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HOW IMPORTANT IS YOUR DATA? 


Years of family photos. Your entire music 
and movie collection. Office documents 
you've put hours of work into. Backups for 
every computer you own. We ask again, how 
important is your data? 


NOW IMAGINE LOSING IT ALL 


Losing one bit - that’s all it takes. One single bit, and 
your file is gone. 





The worst part? You won't know until you | 
absolutely need that file again. Example of one-bit corruption 





THE SOLUTION 


The Mini boasts these state-of-the- 


The FreeNAS Mini has emerged as the clear choice to 
art features: 


Save your digital life. No other NAS in its class offers 


i ry and ZFS bitr 
ECC (error correcting code) memory and ZFS bitrot sieseor Gotti Mora raecseor 


protection to ensure data always reaches disk . Up to 16TB of storage capacity 
without corruption and never degrades over time. - 16GB of ECC memory (with the option to upgrade 
to 32GB) 


, « 2x 1 Gigabit network controllers 
No other NAS combines the inherent data integrity : Ramotemanauementoore (EN) 


and security of the ZFS filesystem with fast on-disk - Tool-less design; hot swappable drive trays 
encryption. No other NAS provides comparable power ISSN re ictal emanecomngured 

and flexibility. The FreeNAS Mini is, hands-down, the 
best home and small office storage appliance you can 
buy on the market. When it comes to saving your 
important data, there simply is no other solution. 
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CERTIFIED 
STORAGE 


With over six million downloads, 
FreeNAS is undisputedly the most 
popular storage operating system 
in the world. 


Sure, you could build your own FreeNAS system: 
research every hardware option, order all the 

parts, wait for everything to ship and arrive, vent at 
customer service because it hasn't, and finally build it 
yourself while hoping everything fits - only to install 
the software and discover that the system you spent 
days agonizing over isn’t even compatible. Or... 


MAKE IT EASY ON YOURSELF 


As the sponsors and lead developers of the FreeNAS 
project, ixsystems has combined over 20 years of 
hardware experience with our FreeNAS expertise to 
bring you FreeNAS Certified Storage. We make it 
easy to enjoy all the benefits of FreeNAS without 
the headache of building, setting up, configuring, 
and supporting it yourself. As one of the leaders in 
the storage industry, you know that you're getting the 
best combination of hardware designed for optimal 
performance with FreeNAS. 


Every FreeNAS server we ship is... 


» Custom built and optimized for your use case 

» Installed, configured, tested, and guaranteed to work out 
of the box 

» Supported by the Silicon Valley team that designed and 
built it 

» Backed by a 3 years parts and labor limited warranty 


eee ee ee 
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As one of the leaders in the storage industry, you 
know that you're getting the best combination 

of hardware designed for optimal performance 

with FreeNAS. Contact us today for a FREE Risk 
Elimination Consultation with one of our FreeNAS 
experts. Remember, every purchase directly supports 
the FreeNAS project so we can continue adding 
features and improvements to the software for years 
to come. And really - why would you buy a FreeNAS 
server from anyone else? 





FreeNAS 1U 

- Intel® Xeon® Processor E3-1200v2 Family 

« Up to 16TB of storage capacity 

* 16GB ECC memory (upgradable to 32GB) 

« 2x 10/100/1000 Gigabit Ethernet controllers 
« Redundant power supply 


FreeNAS 2U 
- 2x Intel® Xeon® Processors E5-2600v2 Family 
« Up to 48TB of storage capacity 
¢ 32GB ECC memory (upgradable to 128GB) 
« 4x 1GbE Network interface (Onboard) - 
(Upgradable to 2 x 10 Gigabit Interface) 
« Redundant Power Supply 











http://www.iXsystems.com/storage/freenas-certified-storage/ 
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EDITOR’S WORD 





Dear Readers, 


# ce to meet you again. Now you are going to read 
the next issue from BSD magazine. You have the 


chance to walk through the installation and the basic 
configuration of Postfix, one of the most popular SMTP servers, 
and SpamAssassin, which will be used for basic e-mail filtering. 





What is more, our experts will show you that most of the 
companies stick to stable/release versions with only security fixes. 
Indeed, if your applications rely on specific API/ABI versions, it is 
better to keep on doing it, but others run experimental branches. 
You will learn more from our article written by David Carlier. 


Finally, you may find interest in the article from the Technologies 
section. The article gives you more insight into industry practices. 


We would like to express our gratitude to our experts who 


contributed to this publication and invite others to cooperate with 
our magazine. 


Enjoy reading, 
Ewa & the BSD team 
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FreeNAS 






IN BUSINESS 


in an Enterprise Environment 


By the time you're reading this, FreeNAS has been downloaded 
more than 5.5 million times. For home users, it’s become an 
indispensable part of their daily lives, akin to the DVR. 
Meanwhile, all over the world, thousands of businesses 
universities, and government departments use FreeNAS to 
build effective storage solutions in myriad applications 


What you will learn... 
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» How TrueNAS builds off the strong points of the FreeBSD and 


FreeNAS operating systems 


* How TrueNAS meets modern storage challenges for enterg 






he FreeNAS operating systems is fre 
) the public and offers thorough doc 
active community, and a feature-rig 
the storage environment. Based on Free 
can share over a host of protocols (SM§ 
FTP, iSCSI, etc) and features an intuiti 
the ZFS file system, a plug-in system 
much more. 
Despite the massive popularity g 
aren't aware of its big brother dut 
data in some of the most demand 
environments: the proven, enterp 
professionally-supported line of, 
But what makes TrueNAS diffd 
Well, I'm glad you asked... 


Commercial Grade Supp 
When a mission critical stor 
organization's whole operat 
halt. Whole community-bag 
free), it can't always get an 
and running in a timely 
responsiveness and expe 
dedicated support tea 
provide that safety. 

Created by the sam 
developed FreeNAS. 
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YOU THIS IMPORTANT ANNOUNCEMENT: 


THE PEOPLE WHO DEVELOP FREENAS, THE WORLD'S MOST 
POPULAR STORAGE OS, HAVE JUST REVAMPED TRUENAS. 
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POWER WITHOUT CONTROL MEANS NOTHING. 
TRUENAS STORAGE GIVES YOU BOTH. 


(Vf Self-Healing Filesystem 
WAMslelPAN EN Elo lis, 
(Vie Qualified for VMware and 


Mi Simple Management 
CAMs esehaecaw\aeal-ciien 
CéMiralilenaeenl nese 


CAN baeriiticcce arene (ve HyperV 
Up Front (no hidden CA Works Great With Citrix 
licensing fees) XenServer® 


To learn more, visit: www.iXsystems.com/truenas 
Adder shi lees rsh fn renee 
ee ge BC Se Eee Ree eee see ee 
VMware and VMware Ready are registered trademarks or trademarks of VMware, Inc. in the United States and other jurisdictions. 
Citrix makes and you receive no representations or warranties of any kind with respect to the third party products, its functionality, the test(s) or the results 
here from, whet = rexpressed, Iimplled, statutery or side 58, saeiaregieats dike Teme aa ati HIM ee eee ee oe jurpose, merchantability, 
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*BSD Corner 
BSD -CURRENT Is Usable Daily Ss 


David Carlier 

Running the development branch of a *BSD daily might sound 
scary. Indeed, this is basically the experimentations’ land and 
this use case seems to apply only to BSD developers — the 
internal APIs might suddenly change because they need to, 
some bugs can be fixed. But some new ones can be introduced 
without notice ... Although that, in general, the community is 
quite reactive and fixes them fairly quickly. David will explain the 
reasons of using what is called the -CURRENT branches. 


Installing the E-mail Servers 
and the Webmail Interface 
Ivan Voras 

The goal of this article is to walk you through the installation and 
the basic configuration of Postfix, one of the most popular SMTP 
servers, and SpamAssassin, which will be used for basic e-mail 
filtering. 
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The Basics of The GDB Debugger -O 


Carlos Neira 

To be able to inspect a program more easily, we need to have the 
symbol table available for the program we intend to debug. This 
is accomplished using the —g flag of the compiler we are going 
to use (we could also debug it without the —g flag but it is really 
cumbersome sometimes). In our case, we will use FreeBSD 10 
as the platform and the clang compiler that comes with it. 


BSD 


MAGAZINE 


Expert Says 


A Complete Guide to FreeNAS 
Hardware Design, Part |: 
Purpose and Best Practices 
Josh Paetzel 

A guide to selecting and building FreeNAS hardware, written 
by the FreeNAS Team, is long past overdue by now. For that, 
we apologize. The issue was the depth and complexity of the 
subject, as you will see by the extensive nature of this four part 
guide, due to the variety of ways FreeNAS can be utilized. 


3G 


Column 


Has the technology sector finally slid into 
the realm of used car salesmen, lawyers 

and ambulance chasers? 38 
Rob Somerville 


Useful Technologies 


Information Security Analytics Finding 
Security Insights, Patterns, 

and Anomalies in Big Data. 
Simulations and Security Processes 
Mark Ryan Talabis, Robert McPherson, 
Inez Miyamoto and Jason L. Martin 
Information Security Analytics gives you insights into the practice 
of analytics and, more importantly, how you can utilize analytic 
techniques to identify trends and outliers that may not be possible 
to identify using traditional security analysis techniques. 
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Running the development branch of a *BSD daily might 
sound scary. Indeed, this is basically the experimentations’ 
land and this use case seems to apply only to BSD developers 
— the internal APIs might suddenly change because they 
need to, some bugs can be fixed, some new ones can 

be introduced without notice (although in general, the 
community is quite reactive and fixes them fairly quickly). 
|am going to talk about the BSDs | know and use the most 
and I'll explain the reasons for using the -CURRENT branches. 


O 


latest version possi- 
ble of clang because | 
am following the com- 
ing of some expected 
features, like OpenMP 
support and_ sanitiz- 
ers support, because 
of the compilation ef- 
fectiveness improve- 
ments, and so on. As 
| often use virtualized 
environments, having 
the latest bhyve fea- 
tures is a very good 
point. From a develop- 
er point of view, hav- 
ing new syscalls like 
explicit_bzero (which 
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ne of the main reasons which | use -CURRENT 
branches is simply having the latest innovations. 
In the case of FreeBSD, | like having the very 


FreeBSD 11.@-CURRENT (IRONFIST) #3 1279098: Sat Feb 21 09:26:20 GMT 2015 





can be preferred in place of memset for some use cas- 
es, avoiding the potential compiler optimization ...), ppoll 
for the Linux emulation layer are beneficial. Casperd pro- 


Welcome to FreeBSD! 


Release Notes, Errata: https: //www.FreeBSD.org/releases/ 

Security Advisories:  https://www.FreeBSD.org/security/ 

FreeBSD Handbook: https: //www. FreeBSD. org/handbook/ 

FreeBSD FAQ: sheet ele leer mele id 

Questions List: https://lists.FreeBSD.org/mailman/ listinfo/freebsd-—questions/ 
FreeBSD Forums: https://forums.FreeBSD.org/ 


Documents installed with the system are in the /usr/local/share/doc/freebsd/ 
directory, or can be installed later with: pkg install en-freebsd-doc 
For other languages, replace "en" with a language code like de or fr. 


Show the version of FreeBSD installed: freebsd-version ; uname -a 
Please include that output and any error messages when posting questions. 
Introduction to manual pages: man man 

FreeBSD directory Layout: elena 


Edit /etc/motd to change this login announcement. 


Figure 1. FreeBSD CURRENT 
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vides some services not available in capsicum’s capabili- 
ties mode and can be seen as a proxy, for example, for 
DNS resolution. 


For OpenBSD, having the latest relayd/httpd features 
interests me (i.e., | run a custom version of relayd which 
produces some additional custom HTTP headers). | ap- 
preciate their “backward compatibility breaking fearless 
for the better good” approach (the recent change in ran- 
dom C API, for example, could confirm it). Indeed since 
the 5.6, the static Position Independent Executable sup- 
port for base system binaries was added, the legacy de- 
terministic rand C API was strongly updated, and so on... 


| recently retried NetBSD, with LLVM/clang in base fol- 
lowing their willingness to move towards it. After some 
days of usage, | noticed a general small performance drop 
(one of my custom applications got something like 5/10 
percent of difference) but it is a generally well Known prob- 
lem with clang; it is improving through the releases. 


Lastly, DragonflyBSD recently brought GCC 5.0 in base 
(with a bunch of new sanitizations flags, in addition to the 
OpenMP 4.0 specifications support). Also more generally, 
a lot of effort is made in the graphic stack. Having the last 
fixes for Hammer 1 filesystem is worthwhile (i.e Hammer2 
is still not production ready). 


>>> World build started on Sun Feb 22 16:52:45 GMT 2015 


oo ORM R A eg Mele Dae | 


>>> Stage 1.1: legacy release compatibility shims 


ce) | ee eee eae] 


me LCE PMO kM M UIT) le) Ret eee he 
===> Lib/clang/libllvmtablegen (obj,depend,all, install) 
===> usr. bin/clang/tblgen (obj,depend,all, install) 

===> usr.bin/clang/clang-tblgen (obj,depend,all, install) 
ee IN Te PARP Cotas) Cm Oe) R 8) ee ene 
ee tase VA Pa lel ee) (Meee See el ee ee 
===> kerberos5/lib/libvers (obj,depend,all, install) 

===> kerberos5/tools/asni_compile (obj,depend,all, install) 
yacc: 4 shift/reduce conflicts. 

ee 1a at Pea ee oe ee ele ee 

aoe Taree eg el ee ee 

===> games/fortune/strfile (obj,depend,all, install) 

===> gnu/usr.bin/gperf (obj,depend,all, install) 

===> gnu/usr.bin/groff (obj,depend,all, install) 





Figure 2. Recompiling FreeBSD... The time needed is fairly variable 
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One of the downsides of running current is if you’re us- 
ing a desktop environment or more generally the ports 
system. In general, when a significant change in the base 
system occurs, it is recommended to rebuild all the ports 
afterwards. The time needed to do so could be poten- 
tially quite important, especially with software like KDE, 
Gnome 3, etc. It is a point to weigh well ... 


For FreeBSD -CURRENT, | very rarely run a desk- 
top. | prefer to use the whole potential CPU/memory for 
compiling the system instead. Also, the fact that | enable 
a significant amount of debugging kernel options which 
slow down the general performance (like WITNESS 
(to detect potential deadlocks) / INVARIANTS (which add 
more kernel level’s assertion) flags) stops me from con- 
sidering it. Those specific options are only useful for de- 
velopers or beta testers though. It is advised to disable 
them otherwise. 


In the case of OpenBSD -CURRENT, | run time in time 
the base cwn which is very light and xorg (called xeno- 
cara) is not in the ports but in the base system, that makes 
those updates easier. In addition, | enable MALLOC _ 
STATS, hence allowing the D flag for MALLOC_OPTIONS 
for debugging purposes with the cost at a performance hit. 
Again, this last one is not recommended if you are not 
a developer. 


yer) ea (GENERIC.MP) #380: Fri Feb 20 20:19:23 GMT 28615 


Welcome to OpenBSD: The proactively secure Unix-Like operating system. 


Please use the sendbug(1) utility to report bugs in the system. 
Before reporting a bug, please try to reproduce it with the latest 
ieee Mm ee | le el 
enough information to reproduce the problem is enclosed, and if a 
known fix for it exists, include that as well. 
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Figure 3. OpenBSD 5.7-BETA, close to the next release 


From a company point of view, if a new feature is gen- 
uinely needed and if it is not possible to do it internally, 
sponsorship might be considered an option. 


Bug acceptability level 

Indeed, the -CURRENT branches introduce potentially some 
new bugs. In the case of FreeBSD, for example, recently the 
Random Number Generator framework change, which was 
made pluggable, was found to be broken. Instead of coming 
back to the previous version, which sounds less risky, the 
issue was fixed — | personally prefer this kind of approach. 
On my side, | run FreeBSD with some local fixes (for bsdgrep, 
for example), some were merged upstream, hopefully some 
others will be in the near future. 
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ec =Werror -Wall -Wimplicit=function-declaration —-Wno=sain -wno=-uninitialized -Wrrame=larger=thane2e4? -semodelekernel =sno0-red=zone =nn0=s5e2 =sno=-55e =ano-Jdnow 
et ee Me ee ae em tL Pd es i ad Ce a oe My DM so i tp Ms ae ey i ee ee ee ae 


ib PMG ee Dae a eta 


=mnd—sax —esott=-float -fno-onit-frame-pointe 
at ie! eee ee ee ne 


Ue A a a a Al ela i i a a MN ec eI vd | 


ee eee eee se eee =0S0CKET_SPLICE -0TCP_SACK -DTCP_ECN -DTCP_ i ee ame ass a | 


ee ee i I Re ee | 


=DAPERTURE -DATRR -ONTFS —0DHIBERNATE Fable ahah ect ia eR ce a hap ad aa oe 


eRe Mt et Oe | 
ec -Werror -Wall -WimpLicit- 
ie PME eae 


fdev/pelsdray radeon; radeon_gart.¢ 


=fno-builtin-wenprint? =-fno-builtin-log 


tg taneh eee de rh ae Se eM ee ae ee pee De dl ly Mee es Mt te UM ieee code My ee Me tpl Ts eiraia 
<fno-builtin-log? -fro-builtin-malloc -fno-ple -02 -pipe -nostdinc -T../../../.. -I. =I. 


Se ec ain oe ee ee eek Le 
farch -DD08 -DOTLACGMOSTIC -0 


or 
KTRACE -DACCOUNTING —DKMEMSTATS -OPTAACE -DPL_DERIMG -DCRYPTO -DS7SVMS6 -OS7SVSEN -057SV5HM —2Ve_SWAP ENCRYPT -OCOMPAT_43 -DFFS -OFFS2 -OFFS_SOFTUPDATES Te hee path ret cal i | 
RAVER -DCOSbb0 -DUDF -DRSDRO0SFS -DFLIFO -DTAPFS -OFUSE -050CKET_SPLICE -01CP_ SACK -DTCP_ECN -DTCP_SIGHATURE -DINET -DINETb -DIPSEL -OPPP_LESOCURP -0PPP_DEFLATE -DPIPEX Panik UE i Mt TE 


Se ee ee ee ees eee meee etme me me ma Rie ao eae ema me a 


eNotes ae KERMEL Swe a ee 

De ee 

cs othe re 
r =fno-builtin- ae La =fno-bul ltin- era 


eee | ee a 


ieee Re a ie ce i es 


eee al arn ete eats De mg i ile Rea nn al tel iach teed ee lM Ce oer Mr Ue i Ue iM Ug ae 
See Rr ee MM ee PM ee Ce IP aera a eaae ee eee =I, =Tiafeafsaf 


“DXEGEMU -DONEWDREVERBOSE -DAULTIPRECESSOR - 


ce EM A Ce dae eS 
PCa Ee 


oor ee ee aa -DPOOL,_ fe eee ee Se a i eae oe en thai) aries al ie ee 
RAVER -DCDS668 -DUDF -DMSDO0SFS -DFIFO -DTMPFS -DFUSE -2S0CKET_SPLICE -O71CP_SACK -OTCP_ECN -DTCP_SIGNATURE -DINET -DINET6 -DIPSEC -DPPP_ESOCOMP -OPPP_DEFLATE -DPIPEX -DMROUTING -DMPLS -OR00T_CONFIG -DUSER_PCICONF 


ce UL es a Um Oat is Ne lem Uy il im i Me 


DMAXUSERS=f8 -O0_KERNEL -4D -HP -¢ Wers.¢ 


ee Me be eee 


ee gem Ete in ete NM ee 


Ce ee ti ee ee ee ee ee ea Mah i rT oh i ee ee A DD 


cot ae Aes =pae felts uta 
ADS ee ce 
a a bed a= a | 


Se dd /bsd 
| 


Figure 4. Recompiling OpenBSD is a quite simple task 
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ea Na ees et ee ee ec) | 





Dragontlame# git diff —-cached > ~/patch-sysctlL.diff 


Figure 5. DragonflyBSD uses git, better for branches handling 


In the case of OpenBSD, the new XHCI driver 
(for USB 3.0) still does not work completely. For example, 
recently a memory leak was found in dhclient (but fixed) ... 
But nothing really major, OpenBSD -CURRENT is runnable 
daily as well. 


DragonflyBSD had memory leaks in the kernel and in 
hammer filesystem. Once again, they were fixed promptly. 


The bug “acceptability” level depends on whether you're 
willing to patiently take the time to make explicit bug re- 
ports in case the bug in question is blocking, or fixing them 
internally and pushing those fixes upstream. But there is 
no support to expect — again a point to consider well. 


Contribution 

Most of the contributions are done in the -CURRENT 
branches. That makes perfect sense as the -CURRENT 
branches are the perfect areas for both fixes and innova- 
tive features, adding disruptive changes whereas the re- 
leases/stables welcome the fixes only. It also makes more 
sense for -CURRENT that recompiling the system is the 
natural usage. 


lf you are a quite advanced BSD user and you wish 
to contribute to make them better for the whole community 
then using the development branches can be considered. 
There are many areas, not only purely technical (like the 
documentation) which can be improved. 


DragonflyBSD uses git internally and due to its branch- 


ing model, it is pretty handy to create a proper diff to sub- 
mit it for review. 
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Conclusion 

Most companies stick to stable/release versions with only 
security fixes. If your applications rely on specific API/ABI 
versions, it is indeed better to keep on doing it. 


Somehow, few others run experimental branches. 
Indeed. For example, Yahoo uses FreeBSD -CURRENT 
internally for their servers. 


Given the short life release cycle chosen by Open- 
BSD with its fair amount of disruptive changes (ie., every 
6 months), it is less surprising to find users using the de- 
velopment branch. 


| recompile quite often FreeBSD / OpenBSD base sys- 
tems but for those who have no interest at all in doing it, 
some snapshots builds are made fairly often ... 


Saying that, it is advised to be registered in the relevant 
mailing lists: freebsd-current@freebsd.org, tech@open- 
bsd.org, tech@netbsd.org, commits@dragonflybsd.org. 





David Carlier has been working as a software developer since 2001. 


He used FreeBSD for more than 10 years and starting from this year, 
he became involved with the HardenedBSD project and performed 
serious developments on FreeBSD. He worked for a mobile product 
company that provides C++ APIs for two years in Ireland. From this, 
he became completely inspired to develop on FreeBSD. 
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Among 
Performance and 
Reliability is critical 
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- ; aS 
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Download syslog-ng Premium Edition 
product evaluation here 


Attend to a free logging tech webinar here 


BalaBit 


IT Security 


www.balabit.com 





syslog-ng log server 


The world's first High-Speed Reliable Logging™ technology 


HIGH-SPEED RELIABLE LOGGING 


m above 500 000 messages per second 





=m zero message loss due to the 
Reliable Log Transfer Protocol™ 


= trusted log transfer and storage 





The High-Speed Reli 
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The goal of this tutorial is to walk readers through 

the installation and the basic configuration of Postfix, 

one of the most popular SMTP servers, and SoamAssassin, 
which will be used for basic e-mail filtering. 


es to and between servers. The delivery of e-mail 

to user-facing software, such as e-mail clients like 
Thunderbird, Outlook and others, is the job of other proto- 
cols like IMAP (and the old, obsolete POP3). 

SMTP was made for a smaller and more trustworthy In- 
ternet and offers next to no guarantees that a message 
was sent from a valid user, to a valid user, or that it will 
arrive in time. However, it offers decent micro-guarantees 
about what happens if a message is received by a serv- 
er. In particular, it offers store-and-forward semantics in 
which the SMTP server receiving a message promises 
that, if it acknowledges that the message was received, it 
has successfully stored the message and will do its best to 
forward it to its intended recipient. The first feature makes 
sending unrequested and forged e-mail very easy (we call 
it soam), and the second feature makes processing e-mail 
fairly resource intensive, as it involved synchronous writ- 
ing of the messages on the server's drives. Because of 
this, running e-mail servers is harder than it should be. To- 
day, SMTP servers are heavily guarded by firewalls, anti- 
viruses and strict rules about who is permitted to send e- 
mail through them. 


S MTP is the protocol used to route e-mail messag- 


SMTP and DNS 

A modern e-mail message has two parts: the username 
and the domain part. We will cover the username part lat- 
er, but the domain part needs some special consideration. 
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As when accessing a web site, an application needs 
to use the DNS system to find out the IP address of the 
server which is accessed. To allow e-mail to be served by 
different servers than those which serve other services 
for the domain, a special type of DNS entry is necessary, 
called the “MX record” (stands for Mail eXchanger). When 
sending e-mail to an address such as “user@example. 
com’, first the MX record for “example.com” is searched. 
If it’s not found, a regular “A” record is searched. 

Note that DNS must be resolvable for e-mail to work, 
as is true with other services, such as the web. 

In addition to MX records, the DNS system can also car- 
ry SPF records (Sender Policy Framework) which can be 
used to inform the world about which IP addresses are al- 
lowed to send e-mail on behalf of which domains. This can 
be used to reduce the possibility of forged e-mails for cer- 
tain domains. 


Complex e-mail routing 

In some cases, the system which receives e-mail for a do- 
main is not the final server which will store the e-mail in the 
users mailboxes. These cases require that e-mail be rout- 
ed from a server to a server, usually from a more general 
server (such as a global organization’s server) to a more 
specific server (Such as a local branch's server). In such 
a scenario, it is possible for the servers which are inter- 
nal to the organization to have local IP addresses, though 
it requires a careful design of the network and its services. 
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E-mail usernames 
On Unix-like systems, the usernames used in e-mail ad- 
dresses are usually the system usernames. In such sys- 
tems, e-mail is delivered to locations provided by the 
system, such as the users’ home directories, and are pro- 
tected by the system access protection rules (such as file 
ownership and access permissions). 

This is not necessarily so: e-mail usernames could be 
stored in a database and delivered to special mailboxes, 
but such setups are outside the scope of this article. 


Spam protection 

Since incoming e-mail messages to an e-mail server are 
unauthenticated and can be easily forged, their content 
needs to be analysed and classified in addition to simple 
checks such as the “To” and “From” addresses. Modern 
anti-spam tools use heuristics and actually process the 
content of the message. Since different people receive 
different types of messages, the best of such systems 
adapt their heuristics to personalize them for each user. 


Disabling Sendmail 

FreeBSD is shipped with Sendmail as the default e-mail 
server system. Sendmail is enough for very simple usage, 
but quickly gets very complicated when additional features 
need to be configured. Since the goal of this tutorial is to in- 
stall Postfix, Sendmail needs to be disabled before Postfix 
can function by adding the following line to /etc/rc. conf: 


sendmail enable=”NONE” 


A reboot is recommended to stop Sendmail listening on 
various network ports. 


Installing and configuring Postfix 
Postfix can be installed from a package with a command 
such as: 


# pkg install postfix 


When asked about activating Postfix in the /etc/mail/ 
mailer.conf, answer a ae 

Postfix configuration files are located in /usr/local/ 
etc/postfix. 

Its main configuration file is main.cf, and this is the file 
which will be modified in the next steps. 

Postfix is very careful about which e-mail to receive, 
and the first line of defense is specifying the domain for 
which it will act as an SMTP end-point server. By default, 
this domain will be extracted from the server's host name, 
specified in the myhostname directive, such as: 
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myhostname = mail.example.com 


The next step is to configure the domains which will be 
accepted as “local” for mail delivery: 


mydestination = Smyhostname, localhost.$Smydomain, 


localhost, Smydomain 


A common configuration is for the SMTP server to be 
configured to accept e-mail from clients in the same lo- 
cal network without authentication. This is convenient 
for the users as it skips the requirement for SMTP login, 
but can escalate into a problem if one of the local ma- 
chines gets taken over by malware which will (ab) use 
the server for sending spam e-mail. 

To configure Postfix to accept connections from IP 
addresses in the same subnets as the server as “trust- 
ed”, configure mynetworks style: mynetworks_ style = 
subnet. 

This method will automatically detect the server's IP 
addresses and subnets. In case manual configuration 
is required, use the mynetworks directive. Some setups 
(for example, servers behind ADSL connections) require 
that e-mail is not routed directly (which is the default be- 
haviour) but is always relayed by the ISP’s e-mail server. 
In this case (and only in this case), use the relayhost di- 
rective to specify the “upstream” e-mail server: 


relayhost = mail.myisp.com 


Such setups usually require that the connection to the 
upstream server is authenticated with the username and 
password provided by the ISP. This can be achieved by 
adding the following lines to the main.cz¢ file: 


smtp sasl auth enable = yes 
smtp sasl password_maps = hash:/usr/local/etc/posttfix/ 
sasl passwd 


Smltp sasl security options = 


The password file specified in the _ 
password maps needs to be created with the following 
content: 


smtp. 6asl 


mail.myisp.com username:password 


This file can contain multiple lines, each specifying 
a server's name and the username and password used 
when connecting to it. Usually, only one line is required. 
Since this is a security sensitive file, you should adjust 
its file access permissions as needed. 
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This text file needs to be converted into Postfix’s “hash- 
map” format by issuing the following command: 


# postmap hash:/usr/local/etc/postfix/sasl passwd 


For historical reasons, the /etc/aliases file needs to be 
parsed and a hashmap file created by issuing the 
newaliases Command: 


# newaliases 


Lastly, Postfix needs to be enabled in /etc/rc.conf 
by adding the following line: 


postfix enable = “YES” 


It can be immediately started by issuing a command 
such as: 


# service postfix start 


Sending an example e-mail message to test Postfix 

The built-in FreeBSD program named “mail” can be used 
to send an example e-mail message. You can specify the 
e-mail Subject value with the “-s” argument, and when the 
program starts, it will read a message directly from the 
console. You should write an example message and end 
it with Ctrl-D: 


> mail -s “A test message” ivoras@example.com 
An example message here. 
[Cpr lp. 


The Ctrl-D keyboard combination will create an “end-of- 
file” signal to the reading program. To check if the e-mail 
was successfully delivered, check the /var/log/maillog 
file and check that a file was created for the user given in 
the above command in the /var/mail directory. 


Installing SoamAssassin 

SpamAssassin is a framework for very configurable and 
adaptable e-mail analysis. It has multiple optional plugins 
which enhance its core functionality. SpamAssassin can 
be installed with the following command: 


# pkg install spamassassin 


lts configuration files are located in /usr/local/etc/mail/ 


Spamassassin. 
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After the installation, you should run the sa-update utility 
to refresh the database of common spam patterns. You can 
do this by adding a line like the following into /etc/crontab: 


* * * * | root /usr/local/bin/sa-update 


Enable the SpamAssassin daemon by adding the follow- 
ing line to /etc/re.conf: 


spamd_ enable="YES” 
Start the daemon with a command such as: 
# service sa-spamd start 


Integrating SpamAssassin with Postfix 

Asmall helper shell script is required to integrate SoamAs- 
sassin with Postfix. For this for this article, we will name 
It /root/spamfilter.sh and give it the following content: 


#!/bin/sh 
SENDMAIL=/usr/local/sbin/sendmail 
SPAMASSASSIN=/usr/local/bin/spamc 

# logger <<<“Spam filter piping 

to SpamAssassin, then to: SSENDMAIL $@” 
S{SPAMASSASSIN} | S{SENDMAIL} “S@” 


exit $? 


Don't forget to make the file executable. 
Next, modify the /usr/local/etc/postfix/master.cf 
file. The first “smtp” line needs to be changed to: 


smtp 
inet. -— - — = smcpd =—o 


content filter=spamfilter 
In addition to this, one new line needs to be added: 


spamfilter 

Unis =. th fi = 

—- pipe 

flags=Rq user=spamd 


argv=/root/spamfilter.sh -oi -f ${sender} ${recipient} 
After these modifications, Postfix needs to be restarted: 
# service postfix restart 

Testing SpamAssassin 


The configuration described in master.cf enables spam 
filtering on the “smtp” service which is bound to the TCP 
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port 25. This means that e-mail sent directly through 
Unix local delivery will not be filtered and SpamAssas- 
sin cannot be tested with the “mail” command. To test 
SpamAssassin, you need to send e-mail through the 
SMTP port 25. 

There is a special string which can be used for test- 
ing. If this string is found in the message by SpamAssas- 
sin, the message will receive 1000 spam points and be 
marked as spam. This string is: 


XJS*C4JDBQADN1 .NSBN3* 2 TDNEN* GTUBE-STANDARD-ANTI-UBE-TEST- 
EMATL*C.34X 


Installing and configuring Dovecot as an IMAP 
server 
IMAP is a protocol for e-mail message retrieval, used by 
user-facing applications to fetch messages and offer them 
in some sort of user interface. In contrast to the old POP3 
protocol, IMAP offers a unified “view” of the message da- 
tabase on the server (messages are usually not deleted 
from the server when retrieved), supports subdirectories 
of the main mailbox and offers some rudimentary ability 
to share mailbox folders between different users. 
Dovecot is one of the most popular IMAP server appli- 
cations. It is well-written and extensible, and cooperates 
well with Postfix. 


How e-mail is delivered 

An SMTP server (operating in one of the roles called a Mail 
Transfer Agent or Mail Submission Agent) accepts a mes- 
sage and then either relays it to another server or attempts 
to deliver it into a “local” mailbox (meaning a mailbox on 
the server where the SMTP server is running). During the 
process of the delivery, the message may be processed 
in a number of ways, for example, by scanning it for spam 
(as seen in the previous tutorial). The delivery is performed 
by a module of the SMTP server called a “delivery agent.” 
A “mailbox” is usually a single text file to which all the e-mail 
is concatenated, together with some special lines which 
delimit it. An alternative standard is called “maildir’ which 
stores each message in its own file, in a special directory 
structure. Once the message is safely written to this stor- 
age, the SMTP server’s job is done. 

Another type of server, for example, the IMAP server, 
implements a protocol by which an application for reading 
e-mail retrieves and presents these messages to the user 
(such applications are called Mail User Agents). The IMAP 
server needs to find the messages and is configured com- 
pletely separately from the SMTP server. In addition to sim- 
ply presenting the messages from the mailbox file through 
the IMAP protocol, Dovecot can perform some interesting 
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additional features, especially when integrated with Post- 
fix. By default, Dovecot will maintain an indexed database 
of messages, which greatly speeds up all operations with 
them, but also contains a specialised delivery agent which 
is smarter than a plain SMTP delivery agent and can filter 
certain types of messages into certain IMAP folders. 


Installing Dovecot 

The current version of Dovecot is available in the pack- 
age named dovecot2, and the Sieve message filtering 
module is in the package named dovecot-pigeonhole. 
Those packages should be installed with the usual pkg 
command: 


# pkg install dovecot2 dovecot-pigeonhole 


lts configuration files’ directory iS /usr/local/etc/ 
dovecot, but the directory is empty after the installation 
and needs to be populated first. The directory: 


/usr/local/share/doc/dovecot/example-config 


contains example configuration files, all of which should 
be copied into Dovecot’s configuration directory (pre- 
serving the directory structure, i.e. the conf.a subdirec- 
tory). In the same way, copy the files from /usr/local/ 
share/doc/dovecot-pigeonhole/example-config/conf.d into 
Dovecot’s conf.d directory. 

For a basic configuration, you should modify the follow- 
ing files, and make sure that the specific configuration 
lines are present and uncommented in them: 


dovecot.conf 
Enable only IMAP with the following line: 


protocols = imap 


conf.d/10-auth.conf 


Disable non-encrypted plaintext logins, include the de- 
fault system authentication mechanism, and specify the 
correct plugins directory: 


disable plaintext auth = yes 
‘include auth-system.conf.ext 
mail plugin dir = /usr/local/lib/dovecot 


conf.d/10-mail.conf 
Specify that the default user mailbox is found in /var/mail, 


but that the additional mailbox files for IMAP folders will 
be in the directory ~/maii for each user separately: 
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mail location = mbox:~/mail: INBOX=/var/mail/%u 


You should also modify the locking methods configuration 
to skip the “dotlock” method which is tricky to use secure- 
ly when the default mailbox is located in /var/mail: 


mbox read locks = fcntl 
mbox write locks = fcntl 


conf.d/10-ssl.conf 


Specify where the TLS certificates are and which cipher 
suite to use (the same ones used for Apache): 


ssl_cert =</var/ssl/ivoras.net.crt 

ssl_key =</var/ssl/ivoras.net.key 

esl Cipher list = 

'ADH: !EXPORT: !SSLv2: EECDHt+aRSA+AESGCM: EECDH+aRSA+RC4:RC4+R 
5A:+HIGH: +MEDIUM: +LOW 

eont.d/1lo-lda. cont 


Specify that the local delivery agent will use the Sieve 
plugin: 


protocol lda { 
mail plugins = $mail plugins sieve 
} 


conf.d/15-mailboxes.conf 


In this file, modify all of the mailbox sections and add 
a line: 


auto = subscribe 
to each uncommented section. 
conf.d/90-sieve.conf 


Enable some convenient Sieve extensions: for modifying 
the e-mail headers and for manipulating IMAP message 
flags: 


Sieve extensions = t+editheader timap4flags 


Using the Dovecot local delivery agent in Postfix 

In order to make use of the Sieve filtering plugin, Postfix 
needs to be configured to pass the e-mail which would be 
delivered locally to Dovecot’s local delivery agent module. 
This is done very simply, with the following line in Postfix’s 
main.cf file: 


mailbox command = /usr/local/libexec/dovecot/dovecot-lda 
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-f “SSENDER” -a 
“SRECIPIENT” 


Restarting Postfix and Dovecot 
Before using Dovecot, enable it in /etc/rc. conf with a line 
such as the following: 


dovecot. enable="ihs” 


Postfix and Dovecot can be restarted by issuing the fol- 
lowing commands: 


# service postfix restart 


# service dovecot restart 


Creating Sieve rules 
Sieve rules can be created globally, for all users, or with 
user-specific scripts. This tutorial will describe the per-us- 
er scenario (for the global case you should look at the 
sieve default directive in 90-sieve.conf). By default, 
Dovecot-Pigeonhole will try to find a Sieve script file 
named ~/.dovecot.sieve in each user’s home directory. 

Sieve has a programming language which is designed 
for simple rule-based e-mail processing. It is a power- 
ful language which can perform many actions, but the 
most common uses of Sieve are for filtering soam mes- 
sages and for sorting e-mail messages into separate 
IMAP folders. 

An example Sieve script can be as follows: 


require [“fileinto”, “envelope”, “imap4flags”, “regex”, 
“editheader”,” variables” ]; 

if header :contains “X-Spam-Flag””YES” { 

fileinto “Junk”; 

} elsif address :contains “to””ivoras@example.com” { 

addheader “Importance”” High”; 

addflag “Slabel3"; 

} elsif address :is “to””mailing-list@example.com” { 

fileinto “mailing-list”; 

} else { 

keep; 

} 


The above script will perform the following actions on 
each message: 


1. If the e-mail headers contain the SpamAssassin’s 
flag, save the message into the “Junk” folder 

2. If the message's “To” header mentions my address 
directly, mark the message as important (this is so 
that messages which contain my address only in the 
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CC field or which are passed through mailing lists are 
marked as “less important”) 

3. If the message is sent to a specific mailing list, save it 
to a separate folder 


Sieve can do much more, and you should study the ex- 
amples given at goo.gl/QGBgDS. 

You can test that everything is working by sending some 
e-mail which will match the above Sieve rules (after re- 
starting Postfix and Dovecot). It is easy to make syntax er- 
rors in Sieve, but luckily the Pigeonhole Sieve module will 
log such errors into a file named ~/.dovecot.sieve.log. 

Be sure to check /var/log/maillog for error messages! 


Connecting to the IMAP server 

When configuring an e-mail reading application, 
you should connect to the IMAP server on the standard 
port 143, and use TLS for secure network traffic, which 
includes logins. 


Installing and configuring the RoundCube 
webmail application 

RoundCube is a “normal” PHP application which offers 
the user the ability to login with a username and password 
to an IMAP server. It collects the messages and folder 
from the IMAP server and displays them in a nice graphi- 
cal interface. It also uses a database for storing miscella- 
neous information such as user preferences and the con- 
tacts list (address book), but the amount of information 
stored to the database is very small (it does NOT store 
e-mail messages in the database; the messages are only 
stored on the IMAP server). 


Installing and configuring RoundCube 
RoundCube requires the following PHP module to be in- 
stalled in addition to those installed for ownCloud: 


php5-filter 

Similarly to how ownCloud was installed in Tutorial #6, this 

tutorial will explain how to install RoundCube from current 

official sources, put into the /srv/www/roundcube directory. 
As a first step, download the source archive from http:// 

roundcube.net/download/ and put it into /srv/www: 


# cd /srv/www 

# fetch 

http://sourceforge.net/projects/roundcubemail/files/ 
roundcubemail/1.0.3/roundcubemail-1.0.3.tar.gz/download 

# tar xzf download 

# mv roundcubemail-1.0.3 roundcube 


# chown -R ivoras roundcube 
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The application needs to write logs and temporary files, 
so the appropriate paths should be allowed to be written 
by the web server: 


# chgrp -R www roundcube/temp roundcube/logs 


# chmod -R gtrw roundcube/temp roundcube/logs 


RoundCube needs to be configured by copying the 
config.inc.php.sample into COnlig-~imc. pnp in the config 
subdirectory, and modifying the following configuration 
variables: 


Sconfig[ ‘db dsnw’] = ‘mysql://roundcube@localhost/ 
roundcube’ ; 

Sconfig[ ‘des key’] = ‘imka9f84mrandomrandom123'; 

sconfig[ ‘mime types’] = ‘/usr/local/etc/apache24/mime.types’ ; 

Sconfig[ ‘default _host’] = ‘tls://localhost’; 


[ 

[ 
Sconfig[ ‘preview pane’] = true; 
Sconfig[ ‘preview pane mark read’] = 2; 
[ 


Ssconfig[ ‘enable installer’] = true; 


The first line configures the database configuration. 
For it to be valid, you should create the roundcube 
database in MySQL and grant the roundcube user all 
rights on it: 


# mysql 


Welcome to the MySQL monitor. Commands end with 
; Or \g. 

Your MySQL connection id is 1 

Server version: 5.5.40 Source distribution 

Copyright (c) 2000, 2014, Oracle and/or its affiliates. 
All rights reserved. 

Oracle is a registered trademark of Oracle Corpora- 
tion and/or its affiliates. Other names may be trademarks 
of their respective owners. 

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current 
input statement. 


mysql> create database roundcube; 
(0.00 sec) 


mysql> grant all on roundcube.* to 


Query OK, 1 row affected 
’roundcube’ @’ localhost’; 


Query OK, 0 rows affected (0.02 sec) 

The second line in the configuration file needs to speci- 
fy a unique key used by RoundCube to securely transmit 
some session-related information. The third line enables 
the built-in installation process, and will need to be re- 


moved before RoundCube is used in production. 
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Next, Apache’s virtual host configuration for the TSL 
host on port 443 needs to be modified to allow access to 
the newly installed PHP application: 


Alias /mail “/srv/www/roundcube” 
<Directory “/srv/www/roundcube”> 
Options ExecCGI FollowSymLinks 
AddHandler fcgid-script php 
FCGIWrapper /usr/local/bin/php-cgi 
aohip 

DirectoryIndex index.php 
AllowOverride All 

Require all granted 


</Directory> 


The reason why it is necessary to allow access to Round- 
Cube only from a SSL-enabled virtual host is because, like 
ownCloud, it requires a login through the web page, so its 
username and password need to be protected. 

RoundCube also requires some configuration changes 
to PHP. By default, there is no php. ini in a freshly installed 
PHP on FreeBSD, but there are two example files named 
php.ini-development @And php.ini-production IN /usr/ 
local/etc. You should copy the production version of the 
file into the php.ini file and change the following lines: 


date.timezone = Europe/Zagreb 
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Next, RoundCube’s internal installation process needs to 
be started by visiting the /installer path in its installa- 
tion, such aS https://example.com/mail/installer. If ev- 
erything is OK, you should arrive at a button to initialize 
the database, which you should click. 


Conclusions 

After database tables are created, you can optionally run 
the tests for the SMTP and IMAP servers (the default con- 
figuration assumes that they are both on localhost), then 
remove the installer line from config.php and the install- 
er directory from RoundCube sources, then visit its main 
URL at https:/example.com/mail. 
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FreeBSD 4.3 and throughout all the versions since. In real life he is a 
researcher, system administrator and a developer, as opportunity 
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ing to cloud computing. He is currently employed at the University of 
Zagreb Faculty of Electrical Engineering and Eomputing and lives in 
Zagreb, Croatia. You can follow him on his blog in English at http:// 
ivoras.net/blog or in Croatian at http://hrblog.ivoras.net/, as well as 
Google+ at https://plus.google.com/+lvanVoras. 
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The Basics of The GDB 
Debugger 


To be able to inspect a program more easily, we need to 
have the symbol table available for the program we intend 
to debug; this is accomplished by using the -g flag of 

the compiler we are going to use (we could also debug it 
without the -g flag but it is really cumbersome sometimes). 
In our case we will use FreeBSD 10 as the platform and the 








clang compiler that comes with it. 


fter a program is compiled using the —g flag we 
Ae: able to peek inside it using the gdb debugger. 

To start a debugging session. All you need to type 
is the following: 


# gdb <program name> 


And we will see a (gdb) prompt. That means that we are 
ready to start typing gdb commands (Figure 1). 

Or if the program we need to debug is currently running, 
we must type: 


#qdb 
#(gdb) attach <pid of running program> 
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Figure 1. GDB example 
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Let's start with some basic commands and _ inspect 
a running application. Dor this example | have selected 
this application http://freeciv.wikia.com/wiki/Main_ Page. 

“Freeciv is a Free and Open Source empire-building 
strategy game inspired by the history of human civili- 
zation. The game commences in prehistory and your 
mission is to lead your tribe from the Stone Age to the 
Space Age...” 

We will inspect the game structures at runtime with gdb. 
Let's follow these steps: 


e Edit /etc/make.conf and add the line witH DEBUG=yes 
(this will not strip your binaries so you will have the 
symbol table and also add the debug flags to the 
compiler when compiling the sources of your ports) 
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Year: 4000 BCE «TO: 
Gold: 50 (*@) 

Tax? @0 Luw: O Seis 60 
(Click For more info> 
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Explorer 
Moves: i 
Plains (BufFalo) 


to the Freeciv version 2.4.35 Server running at 
leg@ged in af '‘créira' conmected to Creira, 


ished cormtroel over the server, You Aave command access 
Players are readys starting game, 


aH A re Oe! bod ee, FD ae ae me ed Dol 
NU gdb 6.1.1 [FreeBSD] 

‘copyright 2004 Free Software Foundatlion, Inc. 

iB 15 free software, covered by the GNU General Public License, and you are 
Lana) in at eS) a ee 
ype “show copying” to see the conditions. 

i a ee i 
ee Se ee ee ee ee ee a ec 

efi 

‘tarting program: fusr/local/bin/freeciv-server 

New LWP 100405] 

New Thread 805406400 (LWP 100405,/Treeciv-server) | 

his 15 the server for Freeciv version 7.4.3 

fou can learn a lot about Freeciv at http: //wew.freeciv.org, 


Tor details. 


freeciv-server program has player authentication support, but it's currently not in us 


Loading rulesets. 

AI*1 has been added as Easy level Al-controlled player 
AI*? has been added as Easy level Al-controlled player 
AI*3 has been added as Easy level Al-controlled player 
AI*4 has been added a5 Easy level Al-control led player 
AI*5 has been added as Easy level Al-controlled player 
Now accepting new client connections 
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Figure 3. The GNU General Public License 
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rogram received signal SIGINT, Interrupt. 
SARS SRM reir te ti OG ee eee tle lee 
C) from /lib/libc.so0.7 
aje|ep eg 
0 Ox0000000801dd606a in select CC 
trea re | ee 
COLO OLE eee a i ad a ee el ce a te 


) from /lib/libc.so.7 


OxOO0O0008008Ta58a in server_sniff_all_input ¢) at sernet.c:686 
OxOO0000080090F407 In Srv_running () at srv_main.c:2317 

eT ET Ee a | 

Ox000000000040276ca in main (argc=1, argqv=O0x7TTttfffdasas) at civserver.c:453 





Figure 4. Jo continue the execution 


¢ Install freeciv from ports Now we will use our first gdb command: 
¢ Start the freeciv server and client (freeciv-server and 

freeciv-gtk2) # gdb /usr/local/bin/freeciv-server 
¢ Join your local game (Figure 2) 
As we don't know anything about how Freeciv works, 
we will press CTRL-C. This will interrupt the program 
and we will take it from there. For starters, let’s interrupt 
and see where we are. If we want to continue the execu- 
tion, we type ‘continue’ or ‘c’ (Figure 4). 

Figure 5 is a screenshot from the client program freeciv- 
gtk2; we need to join our local game as we are going to 
debug the server (Figure 5). 

The #<num> you see are the stackframes of simply called 
frames. When your program is started, the stack has only 
one frame, that of the function main. This is called the initial 
oe frame or the outermost frame. Each time a function is called, 
= ee a new frame is made. Each time a function returns, the frame 





Seem ee erm i 


‘rogram received Signal SIGINT, Interrupt. 
Ue AU UOTE Tele Ceti: Me eo ca er ec 
Cele) ped 
AUR POL ETOTE NOP COR ele eie at ee et 
OP COTO OT COOP UO eee Tr ee 
ORCUTT OO COO | ec te tt 
ie Tel Pee te ee 
OxO0000008008Ta58a in Server_sniff_all_input () at Serneét.c:686 
OxO00000080090T407 In Ssrv_running () at Srv_main.c:2317 
a ey ee eae 
‘6 Ox00000000004076ca in main (argc=l, argv=Ox/Tftttiftdass) at civserver.c:453 
Cel) 
SOULE OLN OL CO ea ee ed er | 2 
aay while (server_sniff_all_input() -=- 5S_E_OTHERWISE) { 
Current language: auto; currently minimal 
(gdb) 11st 


i 
i 


Cece aee e 
check_Tor_full_turn_done(); /* HACK: don't wait during AI phases 
while (server_sniff_all_input() =-- 5_E_OTHERWISE) { 

nothing *, 


L 
J 


*“ After sniff, re-zero the timer: (read-out above on next loop) * 


Ch 


Figure 5. The client program freecivgtk2 
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conn_list_iterate(game.all_ connections, pconn) f{ 
if (srvarg.auth_enabled 
&& !pconn->server.15_closing 
£& pconn->server.status != AS_ ESTABLISHED) { 
Te ee a a pa ee 


} 


He ok ah he 


ee Pe ee ae ee ee i | 
(void) send_server_info_to_metaserver(META_REFRESH): 
eae i ee ee ee ee 


if (fc_select(max_desc + 1, &readfs, &ewritefs, &exceptfs, atv) 


(void) send server_info to metaserver( META REFRESH); 
if (game.info.timeout > 
oo eS dp od 
el i ee 
£& (read timer _seconds(game.server. phase timer) 
> gamé.info.seconds to phasedone)) { 
ee an) 
eae ee 


log_debug("sniffingpackets"): 
check_Tor_Tull_turn_done(); /* HACK: don't wait during AI phases */ 
while (server_sniftTt_all_input¢) == S_E_OTHERWISE) j{ 

coe ee 


After sniff, re-zero the timer: (read-out above on next loop) */ 


Bi 


OxO0000008008fa58a in server_sniff_all_input () at sernet.c:686 | 
if (fc_select(max_desc + 1, &readfs, &writefs, oe 


oe a | 
| om. ae 
eT 
C= 
Fal 


ee ee ae 


a oa coe 


oe ee 


Tire ea ee 
a ee ee ee 
while ¢(server_sniff_all_input() == 5_E_OTHERWISE) 

™ nothing 


| +i 


or 


ieee ee ery 
Paul 
=a) 


After sniff, re-zero the timer: (read-out above on next loop) 


— 
ee 


a: 

: APOE ne: 
ateLal hie) 
(qdb) b sernet.c:695 

Breakpoint 1 at Ox8008Tab1/7: Tile sernet.c, line 695. 
(odb) §j 


ne | 
i] " 
a = 


fas8a in server_sniff_all_input () at sernet.c:686 
q a 


lect(max_desc + 1, &readts, dwritets, &exceptts, &tv) == 0) { 


f 
Lt . 


Figure 7. The innermost frame 
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for that function invocation is eliminated. If a function is re- 

cursive, there can be many frames for the same function. 

The frame for the function in which execution is actually oc- 

curring is called the innermost frame. This is the most recent- 

ly created of all the stack frames that still exist. Let's go into 

frame 3; to do this we type either ‘frame 3’ or ‘f 3’ (Figure 6). 
It seems that the server is going to send us end of turn. 

Let’s make sure to set a break point, the format 

is > 

<break|b> <source.c>:<line number> 

(gdb) b sernet.c:695 

Crit ee 

OP UO N ONT LOO a setts 

Tein ih " a 

Colt) eee 

Ee Ele en 

ee: =- false 


one: 
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in server_sniff_all 
select (max_desc 1 


met 
ee 


faa 
iFbatscte 
Smt ee 

PadTs = : fds_bits = {89, 
aoe __fds_bits {0 
eae ee ee ot. 
LF) 2 a i C¥_uUsSec 


ft 15 T 
ie ita 
he 


<fepeats 
<repeats 16 
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im 
= es 


mes> }} 
i 
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Di 
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fusr/home/cneira/work 


Add sLeed to Aho 


bool add clauses 


eS = 


struct Treaty “ptreaty, 
at cbt His type, 


Tia 


Struct player *pto = (ptrom == ptreaty- 
ce ee i 
struct Clause *pclause; 
enum diplstate_type ds 
= Player_diplstate_get 


pl rd 
ptreaty- 


ees 
a oe es ee es ee | 


pe oe Lil 


ie he a ee 


ae 
Pat) jl 
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a em 
eee eae) Jie ese 
aati FALSE: 


Te 


‘ee Lane eae 


oe 


[a 


mel feels fees fee fe ee ee et i 
i I ea 
2 ee =| | 





a 


(type -- CLAUSE_ADVANCE && 'valid_advance_by_ 


Freabsd- th Thread 8054064 In: sadd_clause 
: ee ey eee 
rae <repeats 15 times>}} 
V_5eC = 1, ogee 
Cereliy) ‘b add clause 
jreakpoint 1 at Ox600cT9436: 
fgdb) c 


SOC ee ee 


T 


Sree ae 


ttt a 
EU 
‘odb) § 


add clause (ptreaty-0x8064b9e600 , 
Bt] 


eet tee. ie 


Figure 10. The add_clause function 
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lt seems we are wrong, let’s interrupt again and inspect 
the data at this point (Figure 8). 

Typing ‘i lo’ means info locals which will display all lo- 
cal variables in this frame and their values, which is pret- 
ty handy. Let's take a look at something easier to see. 
Sometimes in freeciv, another civilization will try to negoti- 
ate terms with us. Looking at the source code, we find the 
add_ clause function in the diptreaty.c source code. That 
function will add a term which will make the other part ac- 
cept or reject our terms (Figure 9 and Figure 10). 

After playing a few minutes we hit this break point. 
At this point, we don’t even know which civilization has 


Sernet.c:686 


ete aie 


oe 


ee ne ee ee 
foe eo 
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Line: 138 PC: oxa00cfo436. 
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approached us to negotiate terms. Now we can know | assume the the negotiation civilization should be in the 
ahead of time as we set the breakpoint where the nego- _— pfrom pointer (Figure12). 
tiation starts (Figure 11). 


ie eri a2 tr a 


bool ‘uct Treaty “ptreaty, 
enum clause ee 


struct player “pto 


[ a a 
Py ‘dip! state_ | 
=- playe Pe a ee ee eee eee ae ea 


| i ‘; ¥PE , i i oe ro. - | ALISE_ | AST) 
re a ee) 


eebsd- th Thread 60540604 In: add_ rains Line: 138 PC: Ox800cTS436 
i en ee ee ke eo) ee ede ee le i 
ana 0, barbarian_type i.) SO love Seeger Ue tO M ROE Be 
was_Created false, 15_connected a a Dl 
gives_shared_vision = {vec = °"\0' erie Claes eee ete eee repeats 200 times=}, 
ie ‘paldimsphuats See ems length ss A ee ee ee ee Ee ant ‘0, length = O}, 
ee cf.) ol a 7400 ""}, time eS a a ee 
D se toy See Cee ee) 
ce eee ae | a ee eee Stat ce el otea tn 
ere: cd | de ee 
tile_wision = {{bits = 1, vec = OxB064d0000 “"}, {bits = 0, vec = OxO}F TTI 





Figure 11. /n the pfrom pointer 


gdb) p ptfrom 
10 = (struct player *) 0x8063dd400 
| 


Figure 12. /n the pfrom pointer 





= : ay 4 
struct player_slot *slot; 
eee ee 
char username[48]: 
char ranked_username[48]: 
Int user_turns: 
_Boo!l 15_male: 
struct government “government; 
struct government *target_government; 
struct nation_type *nation; 
atid aM eee ROG eee aie 


Figure 13. /n the pfrom pointer 





ee a ee ee iT Bie Le Hee ae Te ee 
‘repeats 123 reel ale one perenne. ae ee connected ae 
cCurrent_conn Ux, connectioa 1 sal bee “ist PR iat: ile eee ly Aisa ee hs i Ppe@ats 15 T 
wonders = {0 <repeats 21 times:, 129, 0 <repeats 178 times>}, page te_block = {data = Ox0 
ir ne ef eae al at a =- {data = 0x0, length = Sahm eee he ce 


Wec = OX8O5643800 “"}, Pob = OxX8064dalea0, (Server = {Status ee ee ee 


eh es uel eta ea oe ey ie ‘“\O°" <repeats 15 times>}, debug 
Lt a Ly elt ee a LB Ssagea| ais = {0x80626T000, Oxd, reiaiatal 

tea salto tt eed =) 0 re ee ee a da 47 times>}, client = { 

Cile_vision = {ibIts tn vl a eral ee en oro ie 


ele oe 





Figure 14. /n the pfrom pointer 
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To print the variable’s values, we just type ‘p’. In this case 
‘p is a pointer to a player structure. If we can check the defi- 
nition of the player structure, we just type ptype pfrom and 
the structure definition will be displayed (Figure 13). 

Now let's see what the values are for these fields for the 
demanding civilization. As the pfrom is a pointer we need 
to use pointer notation to check its contents (Figure 14). 


ee ae 


ats 15 times>}, 


Te oa 


ee 
or @g 


Fy) oo ee 
ed ee 


Figure 16. A peace treaty 
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And there we go the full dump for the player struct (Fig- 
ure 15). Looking at the player struct, it seems that the lead- 
er name Is Roy Jenkins and looking at the backtrace (bt), 
the clause of the treaty seems to be “cease fire”, So we are 
going to be offered a peace treaty (Figure 16). To continue 
executing the program type ‘next’ or ‘n’; something like this 
will be displayed in the diplomacy tab (Figure 17). 


eee ee ae i TT 
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J+] Master Jacques de Molay Cy 
| | Gold: | “dh add Clause... 
Eaiae tat (T3) Chief Roy Jenkins 
Gold: 9 = oF Add Clause... 


Figure 17. /n the diplomacy tab 
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freebsd- th Thread 80540604 In: add_clause Line: 143 PC: Oxs00cTS 
| sopra eure sas lel if’ @anhage_avto_explorer (punit-0x805407900) at autoéxplarer.c:39 
do_explore rappel at unittools.c:2447 
Te ed sesgid eae 
srv_@ain () at srv_m 
ER ee gi argv-Ox7 T rtf cCivserver.c: 


Came saved as freeciv-TO009-¥-3550-auto. sav.bz2 


Ereakpoint 2, add_clause (ptreaty=<0x8064b9ee0, pfrom=0x8063dd400, type=CLAUSE_EMBASSY, val=() 
at diptreaty.c:13: 1 


Ce ee ee a a 
struct Clause *pclause-: 
a ee a a 
player_diplstate_get(ptreaty->plr0, ptreaty->plrl1)->type; 


p =e CLAUSE_LAST) { 
Praia aad a eT ee eee 
return FALSE: 


ea5: CLAUSE_ADVANCE && !valid_advance_by a 1 
log_error¢€"Tllegal tech value 41 in clause.", wal): 


eels eo 


(15_ pact_clause(type) 
((ds =|. DOS_PEACE && type = V 
| es Lm i ee CLAUSE oe 
| 


Cen oe ee | nn en 
oe OS_CEASEFIRE && type qe es Sago 


Line: 143 PC: Ox8o00cf9484 
at 
#0 add_clause (ptreaty=0x8004b9ee0, pfrom=0x80630d0d400, type=CLAUSE_EMBASSY, val=0) at diptreaty.c:1: 
rele, evs 
yn 
1151 
p at 
LAUSE EMBASSY 
aan CLAUSE EMBASSY 
= enum clause_type {CLAUSE ADVANCE, CL 
CLAUSE CEASEFIRE, CLAUSE_PEACE, CLAUSE 


Cele 


ih ae ee ee eS 
ALLIANCE, CLAUSE_VISION, CLAUSE_EMBASSY, CLAUSE_LAST} 




















Tea IGE [Vim (a DEV iae lan ie eee Dhan e 
nation_rule_name(nation_of_player(ptreaty-=plri)) 
return FALSE; 


if (type == CLAUSE_EMBASSY 4&& | player has_real_embassy(pto, pfrom)) { 
* we already have embassy 
iog_errar( ITll@gal embassy clause: S&S already fave embassy with Ss. , 
nation_rule_name(nation_of_player(pto)), 
nation_rule_name(nationofplayer(ptrom))): 
return FALSE: 


if €'game.info.trading_gold && type == CLAUSE GOLD) { 
return FALSE; 


if ('game.info.trading_tech && type == CLAUSE_ADVANCE) 
return FALSE: 


= ie i i i i 


it (loate wAtfo.trading_city && type == CLAUSE CITY) { 


Figure 20. The next 
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WE WT ea Cah” sliglutir ae 
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COE CRO T d e 
OxO000000E0090e0084 In Srv_Main () at srv_ 
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Figure 21. The commands 
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Figure 22. The commands 
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What you cannot see in the screenshot is that | have re- Let's go line by line using next; you could also use the 
quested an embassy in return for the cease-fire treaty, but step command but if you use the step command, it will 
here, it is shown on Figure 18. take you inside a function call instead of just evaluating 


sidered. a “a ace ble rebel Bassa 
une rites icing Svea eas EV eis aE struct Senet esa 
ed 

i 

Int total_balance = Q: 

bool only_gitts = TRUE; 

Sa ee see _type ds_after - : 

ee ed ee a ee 
int ec | 


Clause_l1St_iterate(ptreaty--clauses, pclause) | 
ot I 


Se ee ee ee 
ds_after = pact_clause_to_diplstate_type(pclause-> type): 


if (pc lause->type -—- CLAUSE CITY && pclause->from -- pplayer) f 
Given_citiese+: 
} 


clause_list_iterate_end: 


roy pTrany 7 ao Ey 
db) a 
db) 5 
ll_treaty_evaluate (pplayer=-0x806300d400, aplaye 
db) list 
nah 
db) list 
a |e 
i_treaty_evaluate (pplayer=0x8063dd400, aplayer=0x8063 . Ptreaty=0x8064b9ee0) at advdiplomacy.c:578 
db) list 
nay 


Figure 23. The following commands 
aovelp bomacy .c 


ech) ee 
&& city_list_size(pplayer->cities) - given_cities <= 7) 
always keep at least two cities 
DIPLO_LOG(LOG_DIPL2, pplayer, aplayer, “cannot give last cities"): 
cm 


ee) | es 
if (total_balance >= 0) { 
handle_diplomacy_accept_treaty_reg(pplayer, player_number(aplayer)); 
eg ee ee ee ee eee oe 
ee ee ee 
to 
ew irs alan Le a a te eee 
made the proposal. */ 
if (pplayer !'= ptreaty--plro) { 
notify(aplayer, _("*%s (AI)* This deal was not very good for 
player_name(pplayer), 
Jlayer_name(aplayer)); 


freebsd-th Thread 8054064 In: dai_treaty_evaluate Line: 671 PC: Ox80094ac1 
1: total_balance = 0 

(gdb) 11st 

Cele Eee 

Breakpoint 3 at Ox#0094ac13: file advdiplomacy.c, line 621. 

‘Celt |i h ie 

mem ee 


Breakpoint 3, dai_treaty_evaluate (pplayer-0x8063dd400, aplayer-Oxs063dbc00, ptreaty-0x8064b9ee00) 
Pe ie a 
ie ac iil eee 


Figure 24. The commands for program execution 
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the function and returning like the next command 
(Figure 19). 

We are currently at line 143; | just checked what kind of 
data type was CLAUSE EMBASSY, it was a enum one 
(somewhat obvious). Using next a couple of times will get 
us to the next step. See Figure 20. 

Keep on typing ‘n’ and we will exit from the function call and 
arrive to handle diplomacy create clause req (Figure 21 ). 

Let's keep on typing ‘next’ and we will arrive to this func- 
tion call treaty evaluate. That seems interesting. May- 
be the results of rejection or acceptance of conditions are 
done. As | explained earlier, we can step into this one us- 
ing the step command (Figure 22). 

Let's step all the way to get to another point in the program 
execution; after a couple of steps are shown on Figure 23. 

So a quick glance at the source code tells us that the 
total_ balance variable is somewhat important to evaluate 
if a clause is accepted (In our case we are requesting to 
give us an embassy). Instead of printing this variable mul- 
tiple times, let's leave it available in the display. 


#(gdb) display total balance 


Then we set a breakpoint somewhere ahead of advd 
iplomacy.c:621, WE Can SEE that the total _ balance value Is 
displayed and it is -450 (seems bad for our proposal). 

AS we Can Séé@, total balance >=0 Is the condition to 
approve the proposal. This is a review of the commands 
used in this session: 


O Print values and names of all 
local variables in the current 
scope. 


info local 


backtrace bt A backtrare is a summary of 
how your program got where it 
is. It shows one line per fatale, 
for many frames, starting with 
te currently executing frame 
(frame zero), followed by its 


caller (frame one), end on up 


the stock. 
frame f <frame The call stack is divided up 
<frame number> into contiguous pieces called 
number> stack frames, or frames for 


short; each frame is the data 
associated with one call to one 
function. The frame contains 
the arguments. 
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given to the function, the 
function ‘s local variables, 
and address at which the 
function is executing. 


print p <variable> displays the value of the 
<variable> variable 
display disp <variable> Will automatically print the 
<variable> value the variable being 
displayed as long as it is within 
the scope 
win Win Will enter gdb in tui (text user 
interface) mode if we did 
not entered in the first place. 
Default layout is source at the 
top commands at the bottom. 
next n Execute next line of code. Will 
n <number not enter functions. You can 
of next to use as parameter the number 
perform> or times to execute next 
step S Step to next line of code. 
s<number Will step into a function. 
of step to 
Perform> 


These are really basic commands, but really useful. 


Advanced inspection of data structures 
and variables 
Now that we have used the display command or the print 
command, it is getting pretty tedious to manually inspect 
a variable or data structure by typing p or display every 
time we hit a breakpoint we have set. There is a command 
called commands to save us from all this typing. 

First we set a breakpoint where we want to automatically 
inspect data. In this case I'll check one of the city functions. 


(odi5) B-caity.e22352 

(gdb) 4 breakpoint keep y 0x0000000800cflb7b in citizen_ 
base mood at 

ClEy.@r2 352 

Now we can type the following: commands <breakpoint number> 


(gdb) commands 4 


Type commands for when breakpoint 4 is hit, one per line. 
End with a line saying just “end”. 


After you have set the instructions to be executed after 
the breakpoint is hit, you could modify them or just erase 
them like this: 
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(gdb) commands 4 


Type commands for when breakpoint 4 is hit, one per line. 
End with a line saying just “end”. 


> end 

Now if you want to execute something: 

(gdb) commands 4 

Type commands for when breakpoint 4 is hit, one per line. 
End with a line saying just “end”.> printf “Setting city 


mood for leader: %s”, pplayer->name 


> end 


Now we can type all the instructions we want to be execut- 
ed when this breakpoint is hit. Usually, we use print to dis- 
play values, but there is a more powerful function called 
printf that uses a similar format as the C-language function: 


GDB Debugger 





(gdb) printf “Ss”, pplayer->name 
As in C printf, ordinary characters in the template are 
printed verbatim, while conversion specification intro- 
duced by the ‘% character causes subsequent expres- 
sions to be evaluated, their values converted and format- 
ted according to type and style information encoded in 
the conversion specifications, and then printed. 

For example, you can print two values in hex like this: 
printf “foo, bar-foo = 0x%x, 0x%x\n”, foo, bar-foo 
printf supports all the standard C conversion specifica- 
tions, including the flags and modifiers between the ‘%’ 
character and the conversion letter, with the following 
exceptions: 

The argument-ordering modifiers, such as ‘2$’, are not 
supported. 

The modifier ° 
or width. 


“is not supported for specifying precision 
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The “” flag (for separation of digits into groups according 
to LC_NUMERIC’) is not supported. 

The type modifiers ‘hh’, ‘j’, ‘t’, and ‘z’ are not supported. 

The conversion letter ‘n’ (as in “%n’) is not supported. 

The conversion letters ‘a’ and ‘A are not supported. 

Note that the ‘Il’ type modifier is supported only if the un- 
derlying C implementation used to build GDB supports the 
long long int type, and the ‘L’ type modifier is supported 
only if long double type is available. 

As in C, printf supports simple backslash-escape se- 
quences, such as \n, ‘\t’, ‘\V, ‘V’, ‘\a’, and ‘\f, that consist of 
backslash followed by a single character. Octal and hexa- 
decimal escape sequences are not supported. 

Additionally, printf supports conversion specifications 
for DFP (Decimal Floating Point) types using the follow- 
ing length modifiers together with a floating point specifier. 
Letters: ‘H’ for printing Decimal32 types. 

‘D’ for printing Decimal64 types. ‘DD’ for printing Deci- 
mal128 types. 

If the underlying C implementation used to build GDB 
has support for the three length modifiers for DFP types, 
other modifiers such as width and precision will also be 
available for GDB to use. 

In case there is no such C support, no additional modi- 
fiers will be available and the value will be printed in the 
standard way. Here’s an example of printing DFP types 
using the above conversion letters: 


printf “D32: SHf - D64: SDf - D128: SDDF\ 
hn”, l«z2345df,1.2bl0dd,1.Ze1d1 


Dynamically allocated arrays 
Sometimes. it’s better to put most of the type we will 
need to take a look at in the contents of dynamically 
allocated arrays (the ones created by malloc and calloc 
system calls). 

For example we have the usual static memory array: 
Char ¢/ 8001). 
It's easy to display its contents using 
(gdb) pt 
But what about this one: 
Tit: SOP an 
t = (int *) malloc ( 8001 * sizeof( int) ); 


(gdb) pt 


This will give only the address 
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(gdb) p *t 


This will give you the data of the first element in the ar- 
ray, so what is the solution? 


(gdb) p *t@25 


This command will print 25 elements from the array t; the 
format is pointer@<number of elements. 


Getting information from the symbol table 
When we compiled our program with the —g flag, we in- 
structed the compiler to generate a symbol table in our 
program binary. This table contains variable names, func- 
tion names and types. Now let’s suppose we want to know 
the names of all the functions available. We could use one 
of the info family commands: 

(gdb) info functions 

This command will print the names and data types of 
all defined functions. If we want to check only the func- 
tion names matching a regexp we use the command: in- 
fo functions <regexp>. 


For example: 


(gdb) info functions city 
Will match all functions that have city string in their 
name, you must use grep regexp not perl’s regexp. 
The same goes with variables with the command: 
(gdb) info variables 
Print the names and data types of all variables that are 
declared outside of functions (not the local variables). 
Also the same syntax for info variables regexp (gdb) in- 
fo variables city. Print the names and data types of all vari- 
ables (except for local variables) whose names contain 
a match for regular expression regexp. 
(gdb) info address symbol 
Describe where the data for the symbol is_ stored. 
For a register variable, this says which register it is kept in. 
For a non-register local variable, this prints the stack-frame 
offset at which the variable is always stored. Note the con- 
trast with ‘print &symbol’, which does not work at all for 
a register variable, and for a stack local variable prints the 
exact address of the current instantiation of the variable. 
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(gdb) whatis exp 


Print the data type of expression exp. exp is not actual- 
ly evaluated, and any side-effecting operations (Such as 
assignments or function calls) inside it do not take place. 
Any kind of constant, variable or operator defined by the 
programming language you are using is valid in an ex- 
pression in GDB. 


(gdb) whatis 
Print the data type of $, the last value in the value history. 
(gdb) ptype typename 


Print a description of data type typename. typename 
may be the name of a type, or for C code it may have the 
form ‘class class-name’, ‘struct struct-tag’, ‘union union- 
tag’ or ‘enum enum-tag’. 


(gdb) ptype exp 
ptype 


Print a description of the type of expression exp. ptype 
differs from whatis by printing a detailed description, in- 
stead of just the name of the type. For example, for this 
variable declaration: 


struct example {double dtype; float ftype} exl; 
The two commands give this output: 


(gdb) whatis exl 

type = struct example 
(gdb) ptype exl 

type = struct example { 
double dtype; 

float ftype; 

} 


As with what is, using ptype without an argument refers 
to the type of $, the last value in the value history. 


(gdb) info types regexp 


Print a brief description of all types whose name 
matches regexp (or all types in your program, if you 
supply no argument). Each complete typename is 
matched as though it were a complete line; thus, 
‘| type value’ gives information on all types in your 
program whose name includes the string value, but 
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‘i type “value$’ gives information only on types whose 
complete name Is value. 

This command differs from ptype in two ways: first, like 
whatis, it does not print a detailed description; second, it 
lists all source files where a type is defined. 


(gdb) info source 


Show the name of the current source file—that is, the 
source file for the function containing the current point 
of execution—and the language it was written in. (gdb) 
info sources. 

Print the names of all source files in your program for 
which there is debugging information, organized into two 
lists: files whose symbols have already been read, and 
files whose symbols will be read when needed. 


(gdb) Into Tunctions 
Print the names and data types of all defined functions. 
(gdb) info functions regexp 


Print the names and data types of all defined functions 
whose names contain a match for regular expression regexp. 
Thus, ‘info fun step’ finds all functions whose names include 
step; ‘info fun “step’ finds those whose names start with step. 


(gdb) info variables 


Print the names and data types of all variables that are de- 
clared outside of functions (i.e., excluding local variables). 


(gdb) info variables regexp 


Print the names and data types of all variables (except 
for local variables) whose names contain a match for 
regular expression regexp. 


Conclusions 

In GDB we have three ways of interrupting the program 
flow and inspecting what we need; breakpoints, watch- 
points and catchpoints. 

A breakpoint stops the execution at a particular location 
within the program. We have temporary breakpoints, reg- 
exp breakpoints and we could set conditional breakpoints. 

The usual breakpoint : 


(gdb) break <source>:<line> 


(gdb) break <source.c>:<function> 


(gdb) break 3 This one stops at line 3 of the current 
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source file being executed. 


(gdb) break <function> 


The temporary break point is a simple breakpoint that is 
deleted after it is hit; the command for this is: 


(gdb) tbreak <same format as breakpoint> 


The regexp breakpoint sets breakpoints at the functions 
matching the regexp provided 

(gdb) rbreak “*cityConditional breakpoint, stops the 
execution of the program only if the condition is met 


(gdb) b if stremp (commands [0].synopsis,”*start”) == 


Yes, you could use the C library functions as long as 
your program is linked against libc. 

You can enable or disable breakpoints with the follow- 
Ing command: 


enable once — Enable breakpoints for one hit 

enable delete — Enable breakpoints and delete when hit 
(gdb) enable once 1 

(gdb) enable delete 1 


A watchpoint stops the execution when a particular 
memory location (or an expression involving one or more 
locations) changes value. Depending on your system, 
watchpoints may be implemented in software or hard- 
ware. GDB does software watchpointing by single-step- 
ping your program and testing the variable’s value each 
time, which is hundreds of times slower than normal exe- 
cution, but it’s really useful if you really don’t have a clue 
of where the problem is in your program. 
The syntax for this command is: watch <expr> 


(gdb) watch commands [0] 


Watchpoint 1: commands [0] 


A catchpoint stops the execution when a particular event 
occurs. The event could be one of the following. 
Raised signals may be caught: 


Gatch signal — all signals 


catch signal <signame> - a particular signal 
Raised exceptions may be caught: 
¢ catch throw — all exceptions, when thrown 


¢ catch throw <exceptname> — a particular exception, 
when thrown 
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¢ catch catch — all exceptions, when caught 
¢ catch catch <exceptname> — a particular exception, 
when caught 


Thread or process events may be caught: 


¢ catch thread_ start — any threads, just after creation 
¢ catch thread_exit — any threads, just before expiration 
¢ catch thread_join — any threads, just after joins 


Process events may be caught: 


¢ catch start — any processes, just after creation 

¢ catch exit — any processes, just before expiration 
¢ catch fork — calls to fork() 

¢ catch vfork — calls to vfork() 

¢ catch exec — calls to exec() 


Dynamically-linked library events may be caught: 


¢ catch load — loads of any library 

¢ catch load <libname> — loads of a particular library 

¢ catch unload — unloads of any library 

¢ catch unload <libname> — unloads of a particular library 


The act of your program’s execution stopping may also 
be caught: 


¢ catch stop 

¢ C++ exceptions may be caught: 

¢ catch throw — all exceptions, when thrown 
¢ catch catch — all exceptions, when caught 


You can enable and delete breakpoints, watchpoints and 
catchpoints with the enable and delete command. 





Carlos Neira has worked several years asa C/C++ developer and 
kernel porting and debugging enterprise legacy applications. He is 
currently employed as aC developer under Z/OS, debugging and 
troubleshooting legacy applications for a global financial company. 
Also he is engaged in independent research on affective computing. 
In his free time he contributes to the PC-BSD project and enjoys met- 
al detecting. 
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Hardware Design, 


Part I: Purpose and Best Practices 





past overdue by now. For that, we apologize. The issue was the depth and complexity of 

the subject, as you'll see by the extensive nature of this four part guide, due to the variety 
of ways FreeNAS can be utilized. There is no “one-size-fits-all” hardware recipe. Instead, there 
is a wealth of hardware available, with various levels of compatibility with FreeNAS, and there are 
many things to take into account beyond the basic components, from use case and application 
to performance, reliability, redundancy, capacity, budget, need for support, etc. This document 
draws on years of experience with FreeNAS, ZFS, and the OS that lives underneath FreeNAS, 
FreeBSD. Its purpose is to give guidance on intelligently selecting hardware for use with the 
FreeNAS storage operating system, taking the complexity of its myriad uses into account, as well 
as providing some insight into both pathological and optimal configurations for ZFS and FreeNAS. 


f\ guide to selecting and building FreeNAS hardware, written by the FreeNAS Team, is long 


A word about software defined storage 

FreeNAS is an implementation of Software Defined Storage; although software and hardware are 
both required to create a functional system, they are decoupled from one another. We develop and 
provide the software and leave the hardware selection to the user. Implied in this model is the fact 
that there are a lot of moving pieces in a storage device (figuratively, not literally). Although these 
parts are all supposed to work together, the reality is that all parts have firmware, many devices re- 
quire drivers, and the potential for there to be subtle (or gross) incompatibilities is always present. 





Best Practices 

ECC RAM or Not? 

This is probably the most contested issue surrounding ZFS (the filesystem that FreeNAS uses to store 
your data) today. I’ve run ZFS with ECC RAM and I've run it without. I’ve been involved in the FreeN- 
AS community for many years and have seen people argue that ECC is required and others argue 
that it is a pointless waste of money. ZFS does something no other filesystem you'll have available to 
you does: it checksums your data, and it checksums the metadata used by ZFS, and it checksums 
the checksums. If your data is corrupted in memory before it is written, ZFS will happily write (and 
checksum) the corrupted data. Additionally, ZFS has no pre-mount consistency checker or tool that 
can repair filesystem damage. This is very nice when dealing with large storage arrays as a 64TB 
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pool can be mounted in seconds, even after a bad shutdown. However if a non-ECC 
memory module goes haywire, it can cause irreparable damage to your ZFS pool that can 
cause complete loss of the storage. For this reason, | highly recommend the use of ECC RAM with 
“mission-critical” ZFS. Systems with ECC RAM will correct single bit errors on the fly, and will halt the 
system before they can do any damage to the array if multiple bit errors are detected. If it’s imperative 
that your ZFS based system must always be available, ECC RAM is a requirement. If it's only some 
level of annoying (slightly, moderately...) that you need to restore your ZFS system from backups, 
non-ECC RAM will fit the bill. 
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How Much RAM is needed? 

FreeNAS requires 8 GB of RAM for the base configuration. If you are using plugins and/or jails, 
12GB is a better starting point. There’s a lot of advice about how RAM hungry ZFS is, how it requires 
massive amounts of RAM, an oft quoted number is 1GB RAM per TB of storage. The reality is, it’s 
complicated. ZFS does require a base level of RAM to be stable, and the amount of RAM it needs to 
be stable does grow with the size of the storage. 8GB of RAM will get you through the 24TB range. 
Beyond that 16GB is a safer minimum, and once you get past 100TB of storage, 32GB is recom- 
mended. However, that’s just to satisfy the stability side of things. ZFS performance lives and dies 
by its caching. There are no good guidelines for how much cache a given storage size with a given 
number of simultaneous users will need. You can have a 21TB array with 3 users that needs 1GB 
of cache, and a 500TB array with 50 users that need 8GB of cache. Neither of those scenarios are 
likely, but they are possible. The optimal cache size for an array tends to increase with the size of 
the array, but outside of that guidance, the only thing we can recommend is to measure and observe 
as you go. FreeNAS includes tools in the GUI and the command line to see cache utilization. If your 
cache hit ratio is below 90%, you will see performance improvements by adding cache to the sys- 
tem in the form of RAM or SSD L2ARC (dedicated read cache devices in the pool). 


RAID vs. Host Bus Adapters (HBAs) 

ZFS wants direct control of the underlying storage that it is putting your data on. Nothing will make 
ZFS more unstable than something manipulating bits underneath ZFS. Therefore, connecting 
your drives to an HBA or directly to the ports on the motherboard is preferable to using a RAID 
controller; fortunately, HBAs are cheaper than RAID controllers to boot! If you must use a RAID 
controller, disable all write caching on it and disable all consistency checks. If the RAID controller 
has a passthrough or JBOD mode, use it. RAID controllers will complicate disk replacement and 
improperly configuring them can jeopardize the integrity of your volume (Using the write cache 
on a RAID controller is an almost sure-fire way to cause data loss with ZFS, to the tune of losing 
the entire pool). 


Virtualization vs. Bare Metal 

FreeBSD (the underlying OS of FreeNAS) is not the best virtualization guest: it lacks some virtio 
drivers, it lacks some OS features that make it a better behaved guest, and most importantly, it 
lacks full support from some virtualization vendors. In addition, ZFS wants direct access to your 
storage hardware. Many virtualization solutions only support hardware RAID locally (I’m looking 
at you, VMware) thus leading to enabling a worst case scenario of passing through a virtual disk 
on a datastore backed by a hardware RAID controller to a VM running FreeNAS. This puts two 
layers b ZFS and your data, one for the Host Virtualization’s filesystem on the datastore 
and ano the RAID controller. If you can do PCI passthrough of an HBA to a FreeNAS VM, 
and get all the moving pieces to work properly, you can successfully virtualize FreeNAS. We even 
include the guest VM tools in FreeNAS for VMware, mainly because we use VMware to do a lot 
of FreeNAS development. However if you have problems, there are no developer assets running 
FreeNAS as a production VM and help will be hard to come by. For this reason, | highly recom- 
mend that FreeNAS be run “On the Metal” as the only OS on dedicated hardware. 
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Has the technology sector finally slid into 
the realm of used car salesmen, lawyers 
and ambulance chasers? 


Adobe Photoshop inventor Thomas Knoll has made a call for the 
ethical use of the product. Has the technology sector finally slid into 
the realm of used car salesmen, lawyers and ambulance chasers? 


has been a discernible slide from grace over the 

years as to how IT departments, support staff etc. 
have been perceived both by society and management. 
While technology itself may be a factor in this, my per- 
sonal theory is not that technologists are any less mor- 
al per se than any other professional sector, but rather 
when people fear what they do not understand there is an 
instinctive impulse to demonise, categorise, pigeon-hole 
and control. Culturally, we may laugh at the classifications 
of “Geek” and “Nerd”, but ultimately outside of the techni- 
cal community, these are terms of insult rather than en- 
dearment — designed to put the target firmly in their place. 
None is a more hypocritical sight than an HR department 
spewing forth never-ending drivel concerning equality, 
diversity and fairness yet at the same time categorising 
staff in IT as “washing machine engineers”. Apparently, 
IT is now such a demystified specialisation, anyone with 
a screwdriver, a box cutter and a hammer (What — you 
don’t use hammers when upgrading a server?) is suffi- 
ciently competent. Sadly, this devaluation of skills and val- 
ue has permeated management — both middle and senior. 


N SA, security and social media ethics aside, there 


Maybe it is my non-cynical side coming out, but | don't 
believe for a moment that money alone is the main driver 
for career or indeed personal satisfaction for the major- 
ity of people. | would continue to work in IT even | wasn’t 
paid — provided my family and myself had a roof over 
our heads and food in our belly. The lie that we must be 
continual consumers has finally died along with the myth 
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that bankers have integrity, poverty is good for the soul 
and that the Western world we live in is a democracy. 
What drives people is a social contract, that they add val- 
ue by the works of their hands and in return from their 
employer, get little bits of paper or lumps of metal that 
they can exchange for whatever they want. The old model 
stated that we might not give you so many tokens now, but 
we will give you a decent pension, flexible working condi- 
tions, and maybe other benefits such as more holidays 
per year and job stability. This fitted well with a socially 
co-operative model, where the essential servants of so- 
ciety (e.g. Law enforcement, Nurses, Civil servants etc.) 
found a good deal of job satisfaction. If you want more 
benefits, move to a more corporately aggressive sector, 
but in turn you will be expected to compromise your pro- 
fessional integrity more, perform a job you hate, spend 
less time with your family, or just do something morally 
repugnant and hope that your conscience doesn't hurt too 
much. And of course, all with the added benefit of little 
or no job security. The downward spiral of rising costs, 
increased competition and the decreased buying power 
of salaries is finally hitting home across the profession- 
al, skilled and semi-skilled marketplace. The old model 
is dead. The new model demands the commercial stance 
of using technology to cut costs and bring efficiencies and 
it brings with it a very two edged sword. The life of a well- 
designed enterprise scale system can be measured in 
decades, and once the developers, designers, analysts 
and programmers have left, provided the business model 
of the corporate entity does not change much, the return 
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on investment can be huge and the creatives and others 
scaled down. So, in parallel with the false perception that 
anyone can do IT well now that it has become commodi- 
tised, we are heading towards another industrial revolu- 
tion driven by excesses of the military industrial complex. 
Far from learning from the past and using technology to 
bring ethical, moral and social benefit, corporate culture 
has bitten the very hand that feeds it, turned, and start- 
ed to consume its own children. To hell with the human 
and moral consequences, profit and efficiency is all that 
counts and it matters not how many people are put on the 
scrapheap in the process — be they skilled or unskilled. 
And we wonder why the alarm bells are going off by Bill 
Gates, Stephen Hawking and Elon Musk about the dan- 
gers of Artificial Intelligence. 


| don't believe that any creation can be greater than 
its creator. That does not mean that any creation in the 
hands of men cannot pose a significant threat to society. 
In the old days, programmers regularly put back doors in 
their software for maintenance purposes. The kill switch 
was always there. The danger is when we get to the 
stage that neither the end user nor the designer can ef- 
fectively hit the kill switch and this authority is delegat- 
ed to “the system” — and by that | mean either cultural, 
political or a hybrid mix of hardware and software. Case 
in point, we take the mobile phone network for granted 
but in a time of local emergency this can be dedicated to 
Law enforcement etc. cutting off the average consumer. 
While | don't have a problem with this particular scenario 
itself, what would happen if this control was exploited for 
political or economic purposes? Indeed this is happening 
now, but not by the technologists. The HR departments 
first port of call when hiring is Facebook, Twitter or Linke- 
din. All well and good you might think, but what about de- 
cisions being made about your credit worthiness based 
on your associates or lifestyle? Your insurance company? 
The robotic trawling of these sites is a feature of the cor- 
porate landscape, from HR to marketing and reputation 
management. It is not the technology or the technologists, 
itis the power behind the throne, the men in grey suits, the 
hand behind the curtain that truly drives the agenda — and 
those unaccountable faceless corporate clones that either 
wittingly or unwittingly go along with the agenda. 


The irony is that most IT professionals | have encoun- 
tered are a decent bunch whose heart is to improve things 
and make life better, more interesting, more fun. Unfortu- 
nately, we tend to be mesmerised by the environment we 
work in, and forget that outside the IT suite there are those 
— unlike the machines and systems we work with — who 
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have rather dark and ulterior motives. As our culture and 
society becomes more and more globalised — yet ironical- 
ly at the same time more compartmentalised — it is difficult 
to see this as the layers of management and ethical re- 
sponsibility are diluted. Nevertheless, by association, we 
are being tarred with the same brush. Too often, the poli- 
ticians, leaders and the establishment have echoed the 
benefits of change, automation, technology and progress 
but the end result has been far from the idyllic. For the UK, 
in the 80’s we moved to a service rather than a manufac- 
turing based economy. 35 years later there are still areas 
that are blighted by unemployment, especially amongst 
the youth. Be it the Chicago motor industry, shipbuild- 
ing in Glasgow or farming in France, the economic winds 
of change blow, but there remains a fundamental discon- 
nect between the vision, the implementation and the con- 
sequences. 


As technologists we must come to face the harsh truth 
that we fall into the same category as gunsmiths. What we 
design is not bad in itself, it is how it is utilised that mat- 
ters. Sadly, more than ever we better make sure that as 
a profession we are not exploited as useful idiots by those 
that wish to use our creative talents for evil and not for 
good. We urgently need to embrace a strong moral ethic. 
For as history has shown, should we experience a revolu- 
tion on the scale that Gates, Hawking and Musk envisage, 
the creatives, intelligentsia and the useful idiots will be the 
first to face the wrath. 





Rob Somerville has been passionate about technology since his ear- 
ly teens. A keen advocate of open systems since the mid-eighties, he 
has worked in many corporate sectors including finance, automo- 
tive, airlines, government and media in a variety of roles from tech- 
nical support, system administrator, developer, systems integrator 
and IT manager. He has moved on from CP/M and nixie tubes but 
keeps a soldering iron handy just in case. 
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Finding Security Insights, Patterns, and Anomalies 


in Big Data. Simulations and Security Processes 


The primary tool that we will be using in this chapter 

about simulations is Arena, which is commercial software 
developed by Rockwell Automation. Arena is a powerful 
modeling and simulation software allowing a user to model 
and run simulation experiments. We will be using a fully 
functioning perpetual evaluation version, which is available 
for study and download at (http://www.arenasimulation. 
com/Tools_Resources_Download_Arena.aspx). 


is a Windows desktop application, when you start us- 
ing the program, you will see three regions on the main 
Arena window. Let us familiarize you with the three regions: 


. et us get started with simulations. Since Arena 


¢ At the left-hand side of the main window, you will find 
the Project bar containing three tabs: basic process, 
report and navigate panel. In the Project bar, you will 
also find various “Arena modules” to be used when 
building a simulation model. We will discuss more 
about Arena modules in the latter part of this section. 
¢ At the right-hand side, you will find the Model window 
flowchart view to be the largest part of your screen be- 
cause it is your workspace where you will create models. 
You will be creating graphical models using flowcharts, 
images, animations, and other drawn elements. 
At the bottom part of the flowchart view, you will find 
the Model window spreadsheet view, which presents 
all the data associated with the model. 
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This chapter will provide a high-level overview of creat- 
ing simulations in Arena. There are three main steps 
to making a simulation in Arena: 


¢ Design and create a model, 

e Add data and parameters to a model, 
¢ Runa simulation, and 

¢ Analyze a simulation. 


Designing and Creating a Model 

Before we start in Arena, we first need to create a “con- 
ceptual model” for a scenario we will simulate. A con- 
ceptual model is how you think a process should work 
—this could be anything from you just drawing it out on 
a piece of paper or just thinking about it. 

Once you have a conceptual model, the next step is to build 
the model in the workspace using the “modules” in Arena. Mod- 
ules are the building blocks of a model. There are two kinds of 
modules: the flowchart modules and the data modules. 


02/2015 


Information Security Analytics. http://dx.doi.org/10.1016/B978-0-12-800207-0.00004-6 


Copyright © 2015 Elsevier Inc. All rights reserved. 


The flowchart modules illustrate the logic of your simula- 
tion. Some common flowchart modules found in the “Basic 
Process’ tab of the Project bar are the following elements: 
CREATE, PROCESS, DECIDE, DISPOSE, BATCH, SEP- 
ARATE, and ASSIGN and RECORD. To use these mod- 
ules, you simply drag the flowchart module needed into 
the model and then you connect the modules together in 
the Model window flowchart view. For example, if | were to 
create our conceptual model of the IT service desk ticket 
queue, it would look like this (Figure 1). 

As you see in our figure, we used the CREATE, PRO- 
CESS, and DISPOSE modules to illustrate the logic of the 
queue. Once a service desk ticket is created by the IT 

Department (CREATE module), it is processed by the IT 
Department (PROCESS module), and it is closed by the IT 
Department (DISPOSE module). A bit confused? Rest as- 
sured, we have a whole chapter about this and it will get clear- 
er as we take you step by step through an actual scenario. 

For now, we are starting with a three-process scenar- 
io to get you thinking about simulation. This quick start 
model is provided on the companion site for download. 
For now, just think of it as creating a flowchart of your sce- 
nario. If you have used Microsoft Visio before, you will be 
right at home. 


i aie eGo 


IT Processing 
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Figure 1./7 service desk process 


Adding Data and Parameters to the Model 

After creating the flowchart, the next step is to add da- 
ta to each of the flowchart modules. You may assign 
the values for each module by double clicking on the 
modules in the model, which will open up a small dia- 
log window. For example, for the CREATE module, let 
us say tickets arrive at an average of five per hour. 
You would enter that value directly into the CREATE 
module. Additionally, let us say tickets are processed 
and resolved at an average rate of 30 min. You would 
assign this value into your PROCESS module. We will 
provide you with a more detailed walk-through on how 
to do this later in this chapter. 


Running the Simulation 

After the model is complete, all you need to do is to select 
“Go” from the Run menu or press F5. There are other pa- 
rameters that you may want to set up before running the 
simulation, such as the replication parameters where you 
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can set the simulation period. But for the purpose of this 
quick introduction, we will just run the simulation. 


Analyzing the Simulation 

Arena provides a variety of reports so that you may ana- 
lyze the simulation. You access the Reports panel from 
the Project bar. 


CASE STUDY 

There are a lot of interesting uses for simulations in securi- 
ty. One of them is evaluating the effect of security controls 
or mechanisms in your enterprise that otherwise would 
be difficult to recreate. For this chapter, let us put our- 
selves in the position of an Information Security Officer, 
who needs to evaluate different anti-virus (AV) e-mail se- 
curity gateway offerings. One of the main things you will 
be concerned about is performance of the e-mail gateway 
device. Since the device will be sitting in-line and process- 
ing network traffic, you would want to make sure that the 
e-mail gateway is able to handle the volume of e-mails 
coming into your organization. Since this device will also 
sit in front of your e-mail server, there is no convenient 
way to test how the different e-mail security gateways will 
perform. This is where simulations come into play. Simu- 
lations give us a way to predict how a certain scenario or 
situation will play out based on available data. Of course, 
it will not be the same as testing the real thing, but it will 
at least provide us an estimate so that we can make an 
informed decision (Table‘). 

One of the first things we need for a simulation is data. 
Fortunately, in our scenario, a vendor (hereafter referred 
to as Vendor 1) provided us with a data set comparing 
its e-mail security gateway solution with products from 
other vendors (hereafter referred to as Vendor 2 and Ven- 
dor 3). You can download this data set from the book’s 
website. Next, we will explain how this data set will be used 
In our scenario. 


Table 1. Vendor Scenario Data 


Average (s) 0.177271963 0.669560187 0.569069159 

Test data (s) 0.0077 0.0119 0.5994 
0.0018 0.0201 0.5269 
0.0101 3.4405 0.4258 
0.0144 0.0701 0.5109 
0.0134 0.02 0.5619 
0.006 0.0119 0.5017 
0.1103 0.0012 0.4382 
0.0113 0.013 0.4346 
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0.0116 
0.0185 
0.0021 
0.0088 
0.0051 
0.0061 
0.0106 
0.0064 
0.01 
0.0128 
0.0113 
0.01 
0.0058 
0.0023 
0.0126 
0.0128 
0.006 
0.0064 
0.0088 
0.011 
0.0142 
0.0058 
0.0062 
0.0063 
0.014 
0.0946 
0.0011 
0.0073 
0.0089 
0.0111 
0.0081 
0.0114 
0.0096 
0.6305 
0.0113 
0.0059 
0.0102 
0.065 
0.0063 
0.0189 
0.9503 
0.0236 
0.0094 
0.0076 
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0.0161 
0.0157 
0.2894 
0.0089 
0.0056 
0.0067 
0.0206 
0.221 
0.025 
3.067 
1.098 
0.0158 
1.145 
0.112 
0.0146 
0.0098 
0.0201 
0.0139 
0.0066 
1.945 
0.8112 
0.855 
0.874 
1.589 
0.0203 
0.89 
0.0112 
2.547 
3.4003 
2.3314 
0.0158 
0.0144 
0.0204 
0.0061 
0.0105 
2.578 
0.95 
0.721 
3.3614 
0.3078 
3.3444 
0.0103 


0.00254 


1.067 


0.4988 
0.49 
0.4843 
0.4602 
0.4431 
1.4135 
0.4199 
0.4332 
0.4162 
0.4386 
0.4342 
0.4309 
0.4146 
0.4392 
0.4678 
0.4608 
0.4689 
0.481 
0.4449 
0.4312 
0.453 
1.2839 
0.445 
0.4275 
0.4517 
1.092 
0.5119 
0.5966 
1.2248 
0.4345 
0.5527 
0.4991 
0.4213 
1.3264 
0.4312 
0.4246 
0.4422 
1.4509 
0.478 
0.4121 
1.1532 
0.4589 
0.4124 
0.5074 


0.0057 
1.0007 
0.0061 
0.0113 
0.0094 
0.0061 
0.0088 
0.0054 
0.9407 
12.2007 
0.0035 
0.0028 
0.0042 
0.083 
0.0009 
0.0078 
0.0357 
0.0068 
0.0107 
0.0128 
0.0113 
0.9457 
0.0109 
0.0181 
0.0099 
0.0066 
0.0111 
0.0108 
0.0159 
0.0155 
0.0113 
0.0057 
0.0064 
0.0126 
0.0171 
0.0038 
0.0059 
0.0043 
0.0066 
0.0069 
0.01 
0.0064 
0.0119 
0.0113 


0.905 
3.4747 
0.0205 
0.013 
0.0018 
0.0101 
1.345 
0.0936 
3.7085 
3.4655 
1.523 
0.0202 
0.0147 
0.9678 
0.0059 
0.0211 
0.0496 
0.016 
0.0177 
1.8678 
0.013 
0.812 
0.0071 
1.78 
0.0102 
0.832 
0.0127 
0.0144 
0.0026 
0.0772 
0.0136 
0.0101 
0.0125 
0.0146 
0.042 
0.1454 
1.89 
0.0407 
0.8901 
0.8542 
0.0059 
1.956 
1.993 
1.432 


0.4509 
0.4639 
0.4729 
0.4343 
0.4359 
0.4761 
0.4594 
0.6192 
1.1916 
0.5122 
0.4097 
0.4422 
0.4585 
1.282 
1.4524 
0.5503 
0.7331 
0.4823 
0.4378 
0.4388 
0.4349 
0.9953 
0.4457 
0.4099 
0.4278 
0.4231 
0.4346 
0.4988 
1.4738 
0.4918 
0.4157 
0.4327 
0.5496 
0.4308 
0.4525 
0.6053 
0.4243 
0.7431 
0.4764 
0.4635 
0.522 
0.4802 
0.4333 
0.4343 
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0.0111 0.0127 0.4347 
0.0064 0.0125 0.5521 
0.065 0.711 1.2924 
0.0912 0.0056 0.5121 
0.0059 0.0107 0.4661 
0.0125 0.0124 0.4177 
0.0113 0.013 0.4157 
0.8998 1.9081 0.4626 
0.0059 0.0102 0.4304 
0.0184 1.145 0.4216 
0.0099 0.0144 0.5201 





Table 2. Vendor Processing Time — Overall Performance 


Vendor 1 0.177271963 
Vendor 2 0.669560187 
Vendor 3 0.569069159 


Vendor 1 ran malicious e-mails through its e-mail secu- 
rity gateway and computed how fast the gateways pro- 
cessed the malicious e-mails (e.g., how fast the malicious 
e-mails were detected). Since Vendor 1 provided the data, 
as expected in terms of average processing times, Ven- 
dor 1 had an extremely short processing time (Table 2). 

You may be asking yourself, how do we validate these 
numbers? Typically you would just take this data at face 
value and accept these numbers. However, what if you 
wanted to dive deeper to see if these are actually accu- 
rate for your organization’s situation? This is where the 
fun part starts because we can do this through simulation. 
Let us dive into Arena! 

First off, let us deconstruct our scenario. We need three 
components to start our simulation: 


e First, we need to create the e-mails; 

¢ Second, we need to create the ‘e-mail security gate- 
way to process these e-mails; and 

¢« Third, we need to create the inboxes that will receive 
the e-mails. 


Fortunately, creating all of these components is fairly easy 
to do in Arena. Let us start first by creating a stream of e- 
mails that will come into our organization. This can be do- 
ne by using the CREATE module (Figure 2). 

One of the most important things that we need to 
do for a simulation is to create objects that will flow 
through the simulation that we are creating. In our sce- 
nario, the objects flowing through the system are the e- 
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Figure 2. /nserting the CREATE module 


mails that will go through our security devices. In Are- 
na, these objects are known as “entities.” To be able to 
create entities, we need a CREATE module. 

To make a CREATE module, all you have to do is drag 
the icon named create from the left-hand Basic Process 
bar to your work area. Your work area should look similar 
to Figure 3 below. It still looks a little sparse right now but 
this is only our first step. 

Once you have added the CREATE module, the next 
step is to start configuring the attributes and properties for 
that module. To assign value to attributes or properties of 
the module, double click on the CREATE shape so that 
a dialog window appears, as shown in Figure 4. 

In the dialog box, assign any name describing the en- 
tity being created. In this case, we labeled the entities as 
external e-mails. Let us change the entity type to “E-mail” 





Figure 3. Using CREATE to create external e-mail entities 
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as well. We will also tell the simulation the average rate 
of e-mail arrival. The arrival of e-mails could be different 
for each organization. There are different ways of estimat- 
ing this information (i.e., looking through your logs), but 
for the purposes of this example, we shall assume that on 
average an e-mail arrives every second. We can do this 
by changing the following: 


¢ Type: Random (Expo) 
¢ Value: 1 

¢« Units: Seconds 

e Entities per Arrival: 1 
¢ Max Arrivals: Infinite 
¢ First Creation: 0.0. 


At this point, we have created entities for our simula- 
tion. This means that e-mails can now enter our system. 
But where will it go? Right now, nowhere. We need these 
e-mails to be processed, so we will need to create a pro- 
cess. This is done by dragging the PROCESS module 
from the left-hand navigation bar into the workplace as il- 
lustrated in Figure 5. 

Since the e-mails going through the system need to be 
processed by the AV gateway, we will pattern our process 
to our gateway. In our simulation, the PROCESS mod- 
ule will represent the AV gateway that will be processing 
the external e-mails. Similar to what we did with the CRE- 
ATE module, we will configure the attributes and proper- 
ties of the PROCESS module. Open the dialog box for the 
PROCESS module in the same way you did with the CRE- 
ATE module by double clicking on it. 
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Figure 4. Updating the properties of the CREATE module 
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First, let us assign a name for the process module. For 
this example, we will name it “Security Gateway Vendor.” 
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Figure 5. Adding the PROCESS module 


Next we will set the ACTION for this element. In the ac- 
tion drop down, we will select “Seize, Delay, Release” ac- 
tion. This means that when an e-mail arrives, it will wait 
until the resource becomes available and seize the re- 
source, it will wait for the service interval, and then release 
the resource. This is essentially how an e-mail gateway 
operates: before a gateway sends an e-mail to the inbox, 
it will seize, delay (because of processing), and then re- 
lease either to a user’s inbox or a quarantine. 

The “Delay” is an important value here in our simulation be- 
cause it is actually the processing time. Relative to our sce- 
nario, this is the length of time the security gateway takes to 
process an e-mail to find out whether it is malicious or not. 

Our next step is to customize our scenario. Since we 
have the vendor results on the average processing time, 
let us put the average processing value of Vendor 1 for 
this example. Your dialog box should look similar to Fig- 
ure 6 and would have the following parameters: 


¢ Name: Security Gateway Vendor 

¢ Type: Standard 

¢ Action: Seize Delay Release 

¢ Priority Medium 

¢ Resources: Resource, Resource 1,1 
¢ Delay Type: Constant 

¢ Units: Seconds 

¢ Allocation: Value Added 

¢ Report Statistics: Checked 

¢ Value: 0.1777271863. 


The next step is to create the resource for our security 
gateway. Since we are only going to be using one security 
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gateway,v we will only create one resource. This setting 
is important if you are simulating multiple appliances 
or, in other cases, multiple processors. At this point, for 
simplicity, we will only create one resource, which can be 
done by clicking the “Add” button, which is located next 
to the Resource box. Your Resource dialog box should 
have the following parameters: 


¢ Type: Resource 
e Resource: 1 
¢ Quantity: 1. 


As the last step in our PROCESS module, we need to 
ensure that the CREATE module and the PROCESS 
module are linked together. In our scenario, this ensures 
that the e-mails created by the CREATE module goes 
to the PROCESS module to be processed by our AV 
gateway (Figure 7). Typically, Arena does this automatically; 
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Figure 7. Updating the resource property of the PROCESS module 
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however, if it does not, click the Connect button in the upper 
toolbar of Arena to link both modules as seen in Figure 8. 
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Figure 8. Connect the CREATE module with the PROCESS module 


Finally, after processing, we need somewhere for the 
e-mails to go. This is where we use the DISPOSE module. 
Drag a DISPOSE module into your work area and label it 
as “Mailboxes.” Then, connect the PROCESS module with 
the DISPOSE module. This means that after processing, 
the e-mails go to the mailboxes. 

At this point, you are probably thinking that something is 
amiss with this scenario. Why would all processed e-mails 
go directly to the inboxes, right? You are absolutely right 
that something is amiss. For the sake of keeping our step- 
by-step tutorial simple, let us work with what we have for 
now. We will continue to expand on our scenario to make 
it more realistic. Your final simulation should look similar 
to the model in Figure 9. 

Now that we have our simulation model, we are ready to 
run our first simulation. Before running our simulation, you 
will need to configure the different settings for the simu- 
lation. Since a simulation is technically trying to recreate 
a real-world scenario, we need to set up how long and 
how frequently we would like to let the scenario run. 

This is fairly easy to do in Arena. Just click on Run (it is 
a selection on the top bar) and select Run Setup. For this 
simulation let us run it three times for 7 days, 24-hours 
a day. Since e-mails arrive at a one second interval, the 
base time unit will need to be changed to seconds. You will 
see a dialog box similar to Figure 10, in which you will add 
the following parameters: 


¢ Number of Replications: 3 


¢ Initialize Between Replications: Statistics and System 
Checked 
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e Warm-up Period: 0.0 

¢ Replication Length: 7 

¢ Hours Per Day: 24 

¢ Base Time Units: Seconds 

¢ Time Units: Hours 

¢ Time Units: Days 

¢ Termination Condition: Leave Blank. 
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Figure 9. Connect the CREATE module with the PROCESS module 


After the Run parameters have been configured, we will 
add some information on the Project Parameters to de- 
scribe the project by clicking on the Project Parameter tab 
(see below figure). We are now ready to run our simula- 
tion. You do this by simply clicking Run, then selecting Go. 


¢ Project Title: Security Gateway 
¢ Project Description: Add any description 


After clicking Go, the simulation will animate and there 
will be elements moving. You will see “e-mails” coming 
from the CREATE module (external e-mails), moving to 
the PROCESS module (security gateway), and being ac- 
cepted by the DISPOSE module (inbox) (Figure 11). 
Congratulations, you have now completed your first 
simulation! The simulation may take some time to process 
before we get the results. Unfortunately, even with setting 
at the highest speed, running three simulations on e-mails 
over a /-day period will take some time to process. 
Fortunately, Arena has a feature, which allows what is 
called “batch processing.” Batch processing bypasses all 
of the animation, which speeds up processing. To do this, 
you first need to stop the simulation by clicking on Run, 
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Figure 10. Run parameters set up 





and then End. Next, select Run Control and click on Batch 
Run (No Animation). By doing this, you will speed up the 
simulation so that you can generate the results faster (Fig- 
ure 12). 
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Figure 11. Project set up 


Let us try running the simulation again using these new 
parameters. You will notice this time that you see no ani- 
mation and you immediately receive the results. 

Your output will be a report, including some interesting 
values such as minimum averages, maximum averages, 
minimum values, and maximum values. Basically, these 
values are the descriptive statistics for your simulation 
processing times, which we ran three times in 7-day in- 
crements (Figure 13). 
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The real value of the simulation is seen once we start 
comparing the vendors. Let us try doing this next. For each 
vendor, change the delay value to match each vendor's 
average processing time. If you gather the results, you 
should obtain the results in Table 3. Your results should 
show that Vendor 1’s claims are accurate: on average, 
Vendor 1 shows the best performance. 

For now, we will accept that Vendor 1's claim is correct. 
However, as you know, statistics can sometimes be interpret- 
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5 a spread out over a larger range of values. 
a a Let us start with some simple spreadsheet work. 
oa ll a We want to open the file containing the sample data 
ma eae Extemal Emaits } ——*—_—__— Ree ha —— Mailboxes : . 
es i aaa and obtain the SD of the data by using the STDEVP 
a a function (=STDEVP). This explanation may be somewhat 
= 9 Vendor 1 Vendor 2 Vendor 3 
lo Average (sec) 0.177271963 0.669560187 0.569069159 
= Standard Deviation |=STDEVP(D5:D111 0.274832835 
= Test Data (sec) 0.0119 0.5994 
——S SRR 0.0201 0.5269 
Figure 12. A running simulation 3.4405 0.4258 
0.0701 0.5109 
OY Fie ae View Tools mange Chject [Fan] Window Help 0.02 0.5619 
ie er FR » fal Setup | 1+ % Bole He ii HW ® We 0.0119 0.5017 
ce ee ee 0.0012 0.4382 
Saas | 0.013 0.4346 
acer: “ee 0.0161 0.4988 
ff Check btadel Fl 0.0157 0.49 
= am 0.2894 0.4843 
_ 0.0089 0.4602 
= 0.0056 0.4431 
2 0.0067 1.4135 
ies 0.0206 0.4199 
0.221 0.4332 
saan 0.025 0.4162 
3.067 0.4386 
1.098 0.4342 
0.0158 0.4309 
1.145 0.4146 
0.112 0.4392 
0.0146 0.4678 
0.0098 0.4608 
TE 0.0201 0.4689 
— 0.0139 0.481 
| 0.0066 0.4449 
Figure 13. Doing a batch run Figure 14. Computing SD in a spreadsheet 
Table 3. Vendor Processing Time — Initial Simulation Run 
Vendor 1 0.177271963 0.01911149 0.01897183 0.01919783 0.00 0.9207 
Vendor 2 0.669560187 0.6809 0.6760 0.6863 0.00 11.6437 
Vendor 3 0.569069159 0.3770 0.3740 0.3740 0.00 8.3927 
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ed to provide misleading information. Let us start by going 
back and look at the original data that the vendor gave us. 

A very interesting point about the original data is that 
the vendor provided the actual results of their testing. 
With the results for individual processing times for each 
of the e-mails, instead of just the average processing time 
for all of the e-mails, we can use a very simple yet relative- 
ly well-known technique called standard deviation (SD). 

SD shows the variation or dispersion that exists in rela- 
tion to the mean (also called average). A low SD indicates 
the data points tend to be close to the mean (also called 
expected value); a high SD indicates the data points are 
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confusing, so see Figure 14. Open the next tab of the 
sample data containing the computation. 


Table 4. Vendor Processing Time — Adding SD 


Vendor 1 0.177271963 1.185744915 
Vendor 2 0.669560187 1.026043254 
Vendor 3 0.569069159 0.274832835 
Average (s) 0.177271963 0.669560187 0.569069159 
Standard 1.185744915 1.026043254 0.274832835 
deviation 


After computing the SD for all of the vendors, we see 
that Vendor 1 actually has a big SD. This means that the 
results of the test data vary greatly. For example, it pro- 
cesses e-mails very fast in some cases, but in other cas- 
es, it processes e-mails very slowly. You may be asking 
yourself, what exactly does this tell us? Obviously, those 
of you who understand SDs probably already have an in- 
kling of what this means, but we will run a simulation so 
we can see what our scenario generates (Table 4). 

Now, we will go back to the simulation to enter our newly 
computed values. Click on the PROCESS module. How- 
ever, let us change things up a bit. Instead of using the 
“Constant” delay type, we will use the “Normal” delay type 
or what we call a normal distribution. The normal distri- 
bution is a function telling you the probability that an ob- 
servation, in some context, will fall between any two real 
numbers. 

In the PROCESS dialog box, we will maintain the mean 
value, but we will now add the SD for the vendor. The en- 
tries you select should be similar to the below values, 
which will be put in the dialog box (See Figure 15). Next, 
we will run the simulation. 


¢ Name: Security Gateway Vendor 
¢ Type: Standard 

¢ Action: Seize Delay Release 

¢ Priority: Medium 

¢ Resource: Resource, Resource 1,1 
¢ Delay Type: Normal 

e Units: Seconds 

¢ Allocation: Value Added 

¢ Value: 0.177271963 

¢ Std Dev: 1.185744915 

¢ Report Statistics: Checked. 


We will run the simulation for all the vendors and collect 
their results. Remember to make the change to “Normal” 
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for all of the vendors and to add in the SD. Once you 
have run everything and collected the results, your re- 
sults should be similar to values in the tables below. 
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Figure 15. Updating the PROCESS Dialog’s standard deviation 


You should now notice that the results are quite inter- 
esting. The values have changed a considerable amount 
because we used the normal distribution. In fact, Vendor 
1 did not perform as well as expected with these results. 
In this scenario, Vendor 3 actually had better results. 

The reason for this is that Vendor 3 had more consis- 
tent results. The processing times for Vendor 3 were more 
stable and, more importantly, less variable. Conversely, 
Vendor 1 had a lot of variability, which greatly affected 
the overall processing times. This is why it is important to 
understand what you are processing and how it will affect 
your results. Had you just gone with the vendor results, 
you would not have known this information — this provides 
you with value-added information, which could affect your 
choice of a vendor (Table 5). 

For the final part of our tutorial, we will extend our simu- 
lation model to make it more detailed and realistic. In the 
previous scenario, we assumed that all e-mails were ma- 
licious, but in reality we would never do this. For a more 
realistic scenario, we will incorporate the DECIDE module 
to create conditional branches. 

The DECIDE module can be found in the Basic Process 
tab, which is located on the left-hand side of our work ar- 
ea. The DECIDE module helps us to create conditions (al- 
so known as “if-then” conditions) that are similar to what 
you would see in a flowchart (Figure 16). 

We will now create a scenario with conditional 
elements. As we already mentioned, not all e-mails will 
have malicious attachments. Let us say only 5% of all 
e-mails will have malicious attachments. How did we 
get 5%? This is entirely dependent on you, but to have 


02/2015 


Simulations and Security Processes 





Table 5. Vendor Processing Time — Additional SD Settings 


Vendor 1 1.0290 1.0231 1.0342 0.00 25.1238 
Vendor 2 3.8393 3.8183 3.8531 0.00 46.5414 
Vendor 3 0.4661 0.4650 0.4680 0.00 9.4981 
Vendor 1 0.01911149 0.01897183 0.01919783 0.00 0.9207 
Vendor 2 0.6809 0.6760 0.6863 0.00 11.6437 
Vendor 3 0.3770 0.3740 0.3740 0.00 8.3927 


a more realistic scenario, you should probably try to 
check industry benchmarks. For example, Symantec 
issues a monthly intelligence report similar to the one 
in this link where you can find benchmarks: http://www. 
symantec.com/content/en/us/enterprise/other_resources/ 
b-intelligence_report_07-2014.en-us. pdf. 
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¢ Name: Malicious? 
¢ Type: 2-way by Chance 
¢ Percent True (O—100): 5%. 


You should be familiar with the RECORD module, which 
acts like an advance counter. This module is used to run 
different computations and to store the processed re- 
sults within the module. For this scenario, let us make 
a simple counter using the RECORD module to track 
clean and malicious e-mails. If an e-mail is malicious, 
then we will assume the action to be taken is quaran- 
tining the e-mail. If the e-mail is clean, the action to be 
taken is to send it to the user’s inbox. Therefore, we will 
make two counters: one is a Quarantine counter and the 
other is a Mailbox counter. 
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Figure 16. Adding a DECIDE module in the simulation mm 
Now, let us go back to our workspace. Drag a DE- a & 
CIDE module into the work area, and double click on the = 5 
module. Once you are at the dialog box, type in a name sa si 
and change the Type to “2-way chance,” which is the 


default. Since there’s a 5% chance of a mail being ma- 
licious, you will enter 5% in the Percent True text box. 
Your entries should be similar to the parameters shown 
in Figure 17. 

Finally, we will close of the system by adding a DIS- 
POSE module for both the True and False branches. Note 
that all simulations must have a DISPOSE module. Let us 
label the DISPOSE modules as Quarantine for True then 
Mailbox for false. This will be a little understandable when 
we start talking about counters. 


www.bsdmag.org 





Figure 17. Updating the properties of a DECIDE module 


We do this by connecting the RECORD modules in- 
to the DECIDE module, similar to Figure 18. As with all 
simulations, all paths should have an end point. So, you 
need to remember to create DISPOSE modules for the 
two paths: one for the clean e-mails and one for the mali- 
cious e-mails. 
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We will now configure the parameters of our RECORD 
module. Double click on each of the RECORD modules 
and set it to “Count” with a value of 1. This means that if 
the e-mail was malicious, then the RECORD module for 
the Quarantine RECORD module will be increased by 1. 
You would do the same for the Mailbox RECORD module. 
In the context of this scenario, the following applies: if the 
e-mail is malicious (therefore, YES), then the counter for 
malicious e-mails will be incremented by 1. If the e-mail is 
not malicious, then the clean e-mail counter will be incre- 
mented by 1 (Figure 19). 


¢ Name: Malicious E-mail 

¢ Type: Count 

¢ Value: 1 

¢ Record into Set: Unchecked 

¢ Counter Name: Malicious E-mail. 


That is it — our simulation is now complete! We have cre- 
ated a more complex simulation, which utilized the DE- 
CISION and RECORD modules. All you have to do now 
is to run the simulation and wait for the reports to be 
generated (Figure 20). 
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Figure 18. Creating RECORD and DISPOSE modules 


As you can see, our simulation is slowly getting more 
advanced. However, we are still not finished with it. What 
about efficacy? The vendor actually provided us with 
efficacy information and we can use this information to 
improve our simulation. The question is how do we incor- 
porate this information into our simulation (Figure 21)? 

In our previous simulation, we assumed that all e-mails 
that were considered clean were actually clean but in reality, 
things seldom work this way because malicious e-mail will 
get through AV checks. This is why the vendor provided us 
with the ratings of efficacy for each of the products being 
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Figure 19. Updating the properties of the RECORD module 


reviewed. Next, we need to add another conditional element 
in the simulation so that we may include this process. 

We will add another DECIDE module to the second filter 
for the clean decision, but this time, we will add a condi- 
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Figure 21. Additional report information 
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Figure 22. Removing false negatives through another DECISION 
module 


tion for every clean e-mail. In this updated scenario, once 
the decision is made that an e-mail is clean, we place an- 
other decision regarding “how sure are we that the e-mail 
is clean.” We will call this our “True Clean” decision box, 
which is a layer to show the probability that a clean e-mail 
is actually clean. By adding this decision box, we are able 
to provide a means to determine “false negatives” or mali- 
cious e-mails that were missed by the security gateway. 
Your updated simulation should look similar to Figure 22. 


Table 6. Vendor Processing Time — Including Efficacy 


99.90% 





We will now configure our new DECISION module. Dou- 
ble click the True Clean box and add the efficacy rating 
into the Percent True box. This will simulate the probability 
that the e-mail is actually clean. We then add a counter to 
“how many e-mails that were considered clean were actu- 
ally malicious.” We will use the RECORD module to add 
a “Missed Malicious E-mail” box. Below are the e-mails 
that the AV missed, where the vendor’s verdict was clean 
but the e-mails were actually malicious (Table 6). 

Using our vendor spreadsheet, if the vendor’s security 
gateway has a 99.9% efficacy then we put 99.9% in the 
Percent True. These values will allow us to compute the 
probability of an e-mail actually being clean which equates 
to the efficacy in our simulation. See Figure 23. 


¢« Name: True Clean 


¢ Type: 2-way by Chance 
¢ Percent True (O—100): 99%. 
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Finally, let us run our simulation and wait for our report! 
We will do this for all of our vendors. Remember to make 
changes to the average processing times, the SD and 
the efficacy of each simulation. 

As we wrap up this chapter, let us go through the com- 
pleted simulation statistics in Figure 24. In summary, here 
are the observations we obtained from our simulation. 
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Figure 23. Adding efficacy into the simulation 
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Figure 24. Viewing the final report 


¢ Vendor 3 is actually pretty good in terms of 
performance. When we start looking at the efficacy 
(.e., 99.9% vs 98%) and considering the amount 
of e-mails processed in a week, the difference 
between a 99.9% efficacy and a 98% efficacy rate is 
a staggering amount. The difference can be as large 
as 8000 malicious e-mails! 


¢ Even a 99.9% efficacy would result in 568 malicious 


e-mails, which is still a lot of malicious e-mails. This 
shows that even when a vendor’s AV is used, there is 
still a big chance that one of your employees could be 
infected. 
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Vendor 1 1.0290 1.0231 1.0342 
Vendor2 3.8393 3.8183 3.8531 
Vendor3 0.4661 0.4650 0.4680 


The following tables provide a summary of the results we 
collected during the simulation: 


Average Processing Times 


Average Processing 


Vendor 1 0.177271963 
Vendor 2 0.669560187 
Vendor 3 0.569069159 


In this chapter, we demonstrated how it is possible to 
simulate performance when it is difficult to test a system or 
otherwise obtain results. In our security scenario, we sim- 
ulated an AV gateway for three vendors; however, there 
are a lot of other interesting uses for simulations. Another 
possible use of simulations in security could be recreating 


Using a Constant Delay Type 


Vendor 1 0.01911149 0.01897183 
Vendor 2 0.6809 0.676 
Vendor 3 0.377 0.374 

Using a Normal Distribution (STD) 
Vendor 1 0.17727196 
Vendor 2 0.66956019 
Vendor 3 0.56906916 
Vendor 1 1.029 1.0231 
Vendor 2 3.8393 3.8183 
Vendor 3 0.4661 0.465 

Final Results 
Vendor 1 1.029 1.0231 1.0342 
Vendor 2 3.8393 3.8183 3.8531 
Vendor 3 0.4661 0.465 0.468 
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0.00 
0.00 
0.00 


25.1238 604,918 568.33 
46.5414 605,514 1704.00 
9.4981 605,311 9284.00 


virus propagation within a network to see how fast it will 
affect your enterprise. You could also use simulations to 
see the effects of patching, of re-imaging machines and 
of AV updates. On a larger scale, simulations could be 
used to demonstrate cyber attacks against your organiza- 
tion. You can create a simulation representing your whole 
network, including firewalls, a intrusion prevention sys- 
tem and network segments, to see how attacks would or 
would not be detected, among other things. In conclusion, 
simulations in security are particularly useful in evaluat- 
ing the effect of security controls or mechanisms in your 
enterprise that would, otherwise, be difficult to recreate. 


0.01919783 0.9207 
0.6863 11.6437 
0.374 8.3927 

1.18574492 

1.02604325 

0.27483284 
1.0342 25.1238 
3.8531 46.5414 
0.468 9.4981 


25.1238 568.33 
46.5414 1704 
9.4981 9284 
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Data Used in the Simulation 


Average (s) 
Standard deviation 
Test data (s) 
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0.177271963 
1.185744915 


0.0077 
0.0018 
0.0101 
0.0144 
0.0134 
0.006 
0.1103 
0.0113 
0.0116 
0.0185 
0.0021 
0.0088 
0.0051 
0.0061 
0.0106 
0.0064 
0.01 
0.0128 
0.0113 
0.01 
0.0058 
0.0023 
0.0126 
0.0128 
0.006 
0.0064 
0.0088 
0.011 
0.0142 
0.0058 
0.0062 
0.0063 
0.014 
0.0946 
0.0011 
0.0073 
0.0089 
0.0111 
0.0081 
0.0114 


0.669560187 
1.026043254 0.274832835 
0.0119 0.5994 
0.0201 0.5269 
3.4405 0.4258 
0.0701 0.5109 
0.02 0.5619 
0.0119 0.5017 
0.0012 0.4382 
0.013 0.4346 
0.0161 0.4988 
0.0157 0.49 
0.2894 0.4843 
0.0089 0.4602 
0.0056 0.4431 
0.0067 1.4135 
0.0206 0.4199 
0.221 0.4332 
0.025 0.4162 
3.067 0.4386 
1.098 0.4342 
0.0158 0.4309 
1.145 0.4146 
0.112 0.4392 
0.0146 0.4678 
0.0098 0.4608 
0.0201 0.4689 
0.0139 0.481 
0.0066 0.4449 
1.945 0.4312 
0.8112 0.453 
0.855 1.2839 
0.874 0.445 
1.589 0.4275 
0.0203 0.4517 
0.89 1.092 
0.0112 0.5119 
2.547 0.5966 
3.4003 1.2248 
2.3314 0.4345 
0.0158 0.5527 
0.0144 0.4991 


0.569069159 


0.0096 
0.6305 
0.0113 
0.0059 
0.0102 
0.065 
0.0063 
0.0189 
0.9503 
0.0236 
0.0094 
0.0076 
0.0057 
1.0007 
0.0061 
0.0113 
0.0094 
0.0061 
0.0088 
0.0054 
0.9407 
12.2007 
0.0035 
0.0028 
0.0042 
0.083 
0.0009 
0.0078 
0.0357 
0.0068 
0.0107 
0.0128 
0.0113 
0.9457 
0.0109 
0.0181 
0.0099 
0.0066 
0.0111 
0.0108 
0.0159 
0.0155 


0.0204 
0.0061 
0.0105 
2.578 
0.95 
0.721 
3.3614 
0.3078 
3.3444 
0.0103 


0.00254 


1.067 
0.905 
3.4747 
0.0205 
0.013 
0.0018 
0.0101 
1.345 
0.0936 
3./085 
3.4655 
1.523 
0.0202 
0.0147 
0.9678 
0.0059 
0.0211 
0.0496 
0.016 
0.0177 
1.8678 
0.013 
0.812 
0.0071 
1.78 
0.0102 
0.832 
0.0127 
0.0144 
0.0026 
0.0772 
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0.4213 
1.3264 
0.4312 
0.4246 
0.4422 
1.4509 
0.478 
0.4121 
1.1532 
0.4589 
0.4124 
0.5074 
0.4509 
0.4639 
0.4729 
0.4343 
0.4359 
0.4761 
0.4594 
0.6192 
1.1916 
0.5122 
0.4097 
0.4422 
0.4585 
1.282 
1.4524 
0.5503 
0.7331 
0.4823 
0.4378 
0.4388 
0.4349 
0.9953 
0.4457 
0.4099 
0.4278 
0.4231 
0.4346 
0.4988 
1.4738 
0.4918 
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0.0113 0.0136 0.4157 
0.0057 0.0101 0.4327 
0.0064 0.0125 0.5496 
0.0126 0.0146 0.4308 
0.0171 0.042 0.4525 
0.0038 0.1454 0.6053 
0.0059 1.89 0.4243 
0.0043 0.0407 0.7431 
0.0066 0.8901 0.4764 
0.0069 0.8542 0.4635 
0.01 0.0059 0.522 
0.0064 1.956 0.4802 
0.0119 1.993 0.4333 
0.0113 1.432 0.4343 
0.0111 0.0127 0.4347 
0.0064 0.0125 0.5521 
0.065 0.711 1.2924 
0.0912 0.0056 0.5121 
0.0059 0.0107 0.4661 
0.0125 0.0124 0.4177 
0.0113 0.013 0.4157 
0.8998 1.9081 0.4626 
0.0059 0.0102 0.4304 
0.0184 1.145 0.4216 
0.0099 0.0144 0.5201 


Robert McPherson leads a team of data scientists for a Fortune 100 Insur- 
ance and Financial Service company in the United States. He has 14 years 


of experience as a leader of research and analytics teams, specializing in 
predictive modeling, simulations, econometric analysis, and applied sta- 
tistics. Robert works with a team of researchers who utilize simulation 
and big data methods to model the impact of catastrophes on millions of 
insurance policies...simulating up to 100,000 years of hurricanes, earth- 
quakes, and wildfires, as well as severe winter and summer storms, on 
more than 2 trillion dollars worth of insured property value. He has used 
predictive modeling and advanced statistical methods to develop auto- 
mated outlier detection methods, build automated underwriting models, 
perform product and customer segmentation analysis, and design com- 
petitor war game simulations. Robert has a master’s degree in Informa- 
tion Management from the Harvard University Extension. 





|. Miyamoto is a computer investigator in a government agency with 


over 16 years of computer investigative and forensics experience, and 
12 years of intelligence analysis experience. |. Miyamoto is in the process 
of completing a PhD in Systems Engineering and possesses the follow- 
ing degrees: BS in Software Engineering, MA in National Security and 


Strategic Studies, MS in Strategic Intelligence, and EdD in Education. 





Jason L. Martin is Vice President of Cloud Business for FireEye Inc., 
the global leader in advanced threat-detection technology. Pri- 
or to joining FireEye, Jason was the President and CEO of Secure 
DNA (acquired by FireEye), a company that provided innovative se- 
curity products and solutions to companies throughout Asia-Pacif- 
ic and the U.S. Mainland. Customers included Fortune 1000 compa- 
nies, global government agencies, state and local governments, and 
private organizations of all sizes. He has over 15 years of experience 
in Information Security, is a published author and speaker, and is the 
cofounder of the Shakacon Security Conference. 


Information Security Analytics: Finding 
Security Insights, Patterns, and Anomalies 
in Big Data 


by Mark Ryan M. Talabis, Robert McPherson, Inez Miyamoto and Jason L. Martin 


ANALYTICS 


This book provides insights into the practice of analytics and, more importantly, how readers 
can utilize analytic techniques to identify trends and outliers that may not be possible to 
identify using traditional security analysis techniques. It contains information on open-source 
analytics and statistical packages, tools, and applications, as well as step-by-step guidance 

on how to use analytics tools and how they map to the techniques and scenarios provided. 
Readers learn how to design and utilize simulations for «what-if» scenarios to simulate security 
events and processes, and how to utilize big data techniques to assist in incident response and 
intrusion analysis. Written by security practitioners, for security practitioners, the book includes 
real-world_case studies and scenarios for each analytics technique. 
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Using FreeBSD as a File Server with ZFS 


Ivan Voras 


The ZFS storage workshop will teach you how to create a ZFS file system from scratch and build a file server on top 
of it, but it will also teach you how ZFS, file systems and storage servers work in general. You will learn what ZFS 
looks like, its many features and quirks, and how to use it in a FreeBSD server as a building block of a small file 
server. 


ZFS is the ground-breaking file system originally developed at Sun Inc. for their Solaris operating system. It was 
open-sourced as a part of their OpenSolaris initiative and from there has spread to multiple other operating systems. 
FreeBSD was the first one to implement a working port, and though it has taken a fairly long time of tweaking and 
Stabilization, it is now a robust and popular choice. There are products which successfully build upon the technolo- 
gies of FreeBSD and ZFS, such as FreeNAS and its related enterprise-class products from iXsystems, which au- 
tomate and simplify a lot of the tasks, but all of them use the same ZFS interface under the hood, which is not that 
complicated in itself. 


The requirements for this workshop are decent knowledge of FreeBSD, a basic familiarity with command-line op- 
erations, and a system (possibly a virtual machine) on which the student will perform the required tasks, containing 
at least four hard drives (physical or virtual). Since the topic of this workshop is file servers, the participants must 
prepare a virtual or a physical machine with at least two disk drives (and preferably 4), which which to perform the 
exercises and the setup from the workshop. 


http://osdmag.org/course/using-freebsd-as-a-file-server-with-zfs-2/ 


Ivan Voras is a FreeBSD developer and a long-time user, starting with FreeBSD 4.3 and throughout all the versions since. 
In real life he is a researcher, system administrator and a developer, as opportunity presents itself, with a wide range of 
experience from hardware hacking to cloud computing. He is currently employed at the University of Zagreb Faculty of 
Electrical Engineering and Eomputing and lives in Zagreb, Croatia. You can follow him on his blog in English at http:// 
ivoras.net/blog or in Croatian at http://hrblog.ivoras.net/, as well as Google+ at https://plus.google.com/+IvanVoras. 


Our courses are available online in Premium Membership. 
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3 easy steps to optimized checkouts: 


With Gate2Shop, you can optimize An effective payment page variant With dozens of alternative and local 
your payment pages by using testing tool, A/B Testing helps you payment methods offered in 
ready-made templates or by gain insight into user behaviour, multiple currencies, the personal- 
customizing payment pages to your increase payment conversion in the ized checkout allows you to reach 
site look and feel. short and long term. users from all around the world. 


wW Easy integration wW Cross-platform ewWSecure 


2 gate2shop 


Sell. More. 





Call for a free consultation: +44 20 3051 0330 
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